AlgebraicJulia / Catlab.jl

A framework for applied category theory in the Julia language
https://www.algebraicjulia.org
MIT License
599 stars 56 forks source link

Fuzzing / Property-based testing #658

Open kris-brown opened 1 year ago

kris-brown commented 1 year ago

Just leaving this as a potential idea / project for someone to work on: property based testing is very cool.

Properties can be great documentation (e.g. we expect certain invariants, something to be an involution, certain pairs of operations to commute). They allow us to test an efficient implementation of something vs a slower (but more obviously correct one). And even if our functions don't often have interesting properties, just the process of running an algorithm on a bunch of random C-sets (for random C ... or bigger things like random cospans or diagrams of C-sets) and seeing that it doesn't crash would be help for regression testing (this is called fuzzing) - the alternative is hard-coding a particular input to use for a unit test.

The scope of this would be to identify which Julia property-based testing library is best suited for this and then implement the required functionality for C-Sets (e.g. generators, shrinking functions).

epatters commented 1 year ago

This makes a lot of sense, given that a large fraction of the Catlab tests are just checking that certain equational axioms hold. Moreover, those equations are often explicitly recorded in a GAT, so in principle are available to be used.