search

jgm / djot

A light markup language
https://djot.net
MIT License
1.66k stars 43 forks source link

Full coverage tests #107

Closed matklad closed 1 year ago

matklad commented 1 year ago

Looking at our fuzz failures like https://github.com/jgm/djot/issues/106, I think it would be fun to arrange "full-coverage" tests. Basically, theer's only a handful of characters djot recognizes as special [ -*|[]{}()_`:a]. So we can do the following: select every subset of 4 characters, and generate every string of length five, for 1_397_760 tests.

cc https://matklad.github.io/2021/11/07/generate-all-the-things.html

jgm commented 1 year ago

Feel free to play around with fuzz.lua. The code is pretty simple. Note that in generating the random strings, I already privilege the special characters (assigning them higher probability). But getting full coverage on small strings is a nice idea.

jgm commented 1 year ago

There is also room to improve the failure minimization code. Currently I just subtract from beginning and end, but another transformation to try would be removing all non-special characters, removing all non-ascii characters, and/or collapsing adjacent non-special chars.