Edit on GitHub

Reference

Quickcheck

JCheck.Quickcheck — Type
Quickcheck

Contain a set of properties to check through the generation of random input.

Fields

  • description::AbstractString: description for the instance
  • rng::AbstractRNG: PRNG used to generate inputs
  • predicates::PredsAssoc: predicates to check
  • variables::ArgsDict: arguments used by the predicates
  • n::Int: number of random inputs to generate
  • serialize_fails::Bool: if true, serialize failing inputs to a JLSO file
source
JCheck.Quickcheck — Method
Quickcheck(desc; rng=GLOBAL_RNG, n=100, serialize_fails=true)

Constructor for type Quickcheck.

Arguments

  • desc::AbstractString: description for the instance
  • rng::AbstractRNG: PRNG used to generate inputs
  • n::Int: number of random inputs to generate
  • serialize_fails::Bool: if true, serialize failing inputs to a JLSO file

Examples

julia> qc = Quickcheck("A Test")
A Test: 0 predicate and 0 free variable.
source
JCheck.@add_predicate — Macro
@add_predicate qc desc pred

Add the predicate pred to the set of tests qc with description desc.

Arguments

Notes

The form of pred is very strict:

  • It has to be an anonymous function. Formally, it should be an Expr of type ->.
  • The type of each argument appearing on the left-hand side of pred has to be specified with the x::Type syntax.
  • The names of the arguments of pred matter! Specifically, in a given Quickcheck object, the type of every argument must be consistent across predicates (see examples).
  • Each predicate stored in a given Quickcheck object must be given a distinct description.

Examples

julia> qc = Quickcheck("A Test")
A Test: 0 predicate and 0 free variable.

julia> @add_predicate qc "Identity" (x::Float64 -> x == x)
A Test: 1 predicate and 1 free variable.
x::Float64

julia> @add_predicate qc "Sum commute" ((n::Int, x::Float64) -> n + x == x + n)
A Test: 2 predicates and 2 free variables:
n::Int64
x::Float64

julia> @add_predicate qc "Is odd" isodd(x)
ERROR: Predicate declaration must have the form of an anonymous function (... -> ...)
[...]

julia> @add_predicate qc "Is odd" (x::Int -> is_odd(x))
ERROR: A declaration for variable x already exists with type Float64; please choose another name for x
[...]
source
JCheck.@quickcheck — Macro
@quickcheck qc [file_id::AbstractString="yyyy-mm-dd_HH-MM-SS"]

Check the properties specified in object qc of type Quickcheck.

If qc.serialize_fails is true, serialize the failing cases to JCheck_<file_id>.jchk. Those can latter be analyzed using load and @getcases.

Note

If no argument file_id is passed, defaults to current time.

source

Failed Cases Analysis

JCheck.load — Method
load(io)

Load a collection of failed test cases serialized by a @quickcheck run. Argument io can be of type IO, AbstractString or AbstractPath.

Examples

julia> ft = JCheck.load("JCheck_test.jchk")
2 failing predicates:
Product commute
Is odd
source
JCheck.@getcases — Macro
@getcases ft, desc...

Get the predicate with description desc and the valuations for which it failed.

Note

The predicate with description closest to the one given (in the sense of the Levenshtein distance) will be returned; there is no need to pass the exact description.

Examples

julia> ft = JCheck.load("JCheck_test.jchk")
2 failing predicates:
Product commute
Is odd

julia> pred, valuations = @getcases ft iod
NamedTuple{(:predicate, :valuations), Tuple{Function, Vector{Tuple}}}((Serialization.__deserialized_types__.var"#3#4"(), Tuple[(0,), (-9223372036854775808,), (6444904272543528628,)]))

julia> map(x -> pred(x...), valuations)
3-element Vector{Bool}:
 0
 0
 0
source

Random Input Generation

JCheck.generate — Function
generate([rng=GLOBAL_RNG], T, n)

Sample n random instances of type T.

Arguments

  • rng::AbstractRNG: random number generator to use
  • T::Type: type of the instances
  • n::Int: number of realizations to sample

Default generators

generate methods for the following types are shipped with this package:

  • Subtypes of AbstractFloat
  • Subtypes of Integer except BigInt
  • Complex{T <: Real}
  • String
  • Char
  • Array{T, N}
  • BitArray{N}
  • SquareMatrix{T}.
  • Any special matrix implemented by Julia's LinearAlgebra module
  • Union{T...}
  • UnitRange{T}, StepRange{T, T}

In the previous list, T represents any type for which a generate method is implemented.

Special Matrices (LinearAlgebra)

  • Generators are implemented for <...>Triangular{T} as well as <...>Triangular{T, S}. In the first case, S defaults to SquareMatrix{T}. The exact same thing is true for UpperHessenberg.
  • Same idea for <...>diagonal{T, V} with V defaulting to Vector{T}.
  • Generators are only implemented for Symmetric{T, S} and Hermitian{T, S} right now. Most of the time, you will want to specify S = SquareMatrix{T}.

Arrays & Strings

General purpose generators for arrays and strings are a little bit tricky to implement given that a length for each sampled element must be specified. The following choices have been made for the default generators shipped with this package:

  • String: The length of each string is approximately exponentially distributed with mean 64.
  • Array{T, N}: The length of each dimension of a given array is approximately exponentially distributed with mean 24 ÷ N + 1 (in a low effort attempt to keep the number of entries manageable).

If this is not appropriate for your needs, don't hesitate to reimplement generate.

Implementation

When implementing generate for your type T keep the following in mind:

  • Your method should return a Vector{T}.
  • It is not necessary to write generate(T, n) or generate([rng, ]Array{T, N}, n) where N; this is handled automatically. You only need to implement generate(::AbstractRNG, ::Type{T}, ::Int).
  • Consider implementing specialcases and shrink for T as well.

Examples

using Random: Xoshiro

rng = Xoshiro(42)

generate(rng, Float32, 10)

# output

10-element Vector{Float32}:
 -1.5388016f7
 -5.3113024f-19
 -1.3960648f35
  1.5957566f31
 -4.381218f26
  2.380078f35
  3.878954f9
 -1.1950524f-7
  7.525897f24
 -3.1891005f-12
source
JCheck.specialcases — Method
specialcases(T)

Non-random inputs that are always checked by @quickcheck.

Arguments

  • T::Type: type of the inputs

Implementation

  • Your method should return a Vector{T}.
  • Useless without a generate method for T.
  • Be mindful of combinatoric explosion! @quickcheck generate an input for each element of the Cartesian product of the special cases of every argument of the predicates it is trying to falsify. Only include special cases that are truly special.

Examples

julia> specialcases(Int)
4-element Vector{Int64}:
                    0
                    1
 -9223372036854775808
  9223372036854775807

julia> specialcases(Float64)
4-element Vector{Float64}:
   0.0
   1.0
 -Inf
  Inf
source

Shrinkage

JCheck.shrink — Method
shrink(x)

Shrink an input. The returned value is a Vector with elements similar to x. Returning a vector of length 1 is interpreted as meaning that no further shrinkage is possible.

Default Shrinkers

shrink methods for the following types are shipped with this package:

  • AbstractString
  • AbstractArray{T, N} for any T and N

Implementation

  • Any implementation of shrink(x::T) must come with an implementation of shrinkable(x::T). Failure to do so will prevent @quickcheck from calling shrink on an object of type T.
  • shrink(x) must return [x] if shrinkable(x) evaluate to false. We suggest that the first line of your method is something like:
shrinkable(x) || return typeof(x)[x]
source
JCheck.shrinkable — Method
shrinkable(x)

Determines if x is shrinkable.

Note

Shrinkage can easily be disabled for type T using overloading:

shrinkable(::T) = false
source