This PR hooks up the Hypothesis testing library, which enables property-based tests:
For all data matching some specification.
Perform some operations on the data.
Assert something about the result.
As the starline parser has historically been less than robust, I started there.
The specification was: Given some starline-regex matching string, it either is cleanly rejected by parsing, or results in a well-formed Star object. I let that rip, and smoke poured out of latent weak spots and edge cases in parsing.
I then tried another test: Given a regex-matching string resulting in a Star object when parsed, that Star should parse cleanly to an input line and back to the same Star object, which should then parse to the same input line - ie, checking that functional round trips (once initially parsed) hold. More smoke poured out of parsing.
In order to make the starline parser a little less un-robust, I compiled whitelists of star types, trade codes, and trade-code occur-in-same-star pairs from the combined TravellerMap data as at Sep 16, 2023, then applied those.
Not quite believing the effectiveness and thoroughness of the starline-parser test, I had another go on sector object generation.
Still not quite believing the results, I threw Hypothesis at wringing out the canonicalisation pass of delta debugging.
Now I'm convinced.
Looking at test_parse_line_to_star_and_back , @tjoneslo , you'll see two types of @example decorators.
The larger class, such as '0101 000000000000000 ???????-? 000000000000000 - - 0 000 0000V' are inputs that Hypothesis found and reduced, that previously violated that test's specification. I've left them there as regression tests.
The smaller class, such as '2405 Uurku B986445-9 Ni Pa (Ashdak Meshukiiba) Sa { 0 } (A33-2) [2437] Bc - - 222 8 ImDv K1 V' are things I tripped over when running the parsing part of a Charted Space run, that also violated that test's specification. Likewise, I've left them there as regression tests.
This PR hooks up the Hypothesis testing library, which enables property-based tests:
As the starline parser has historically been less than robust, I started there. The specification was:
Given some starline-regex matching string, it either is cleanly rejected by parsing, or results in a well-formed Star object. I let that rip, and smoke poured out of latent weak spots and edge cases in parsing.I then tried another test:
Given a regex-matching string resulting in a Star object when parsed, that Star should parse cleanly to an input line and back to the same Star object, which should then parse to the same input line- ie, checking that functional round trips (once initially parsed) hold. More smoke poured out of parsing.In order to make the starline parser a little less un-robust, I compiled whitelists of star types, trade codes, and trade-code occur-in-same-star pairs from the combined TravellerMap data as at Sep 16, 2023, then applied those.
Not quite believing the effectiveness and thoroughness of the starline-parser test, I had another go on sector object generation.
Still not quite believing the results, I threw Hypothesis at wringing out the canonicalisation pass of delta debugging. Now I'm convinced.
Looking at
test_parse_line_to_star_and_back, @tjoneslo , you'll see two types of@exampledecorators.The larger class, such as
'0101 000000000000000 ???????-? 000000000000000 - - 0 000 0000V'are inputs that Hypothesis found and reduced, that previously violated that test's specification. I've left them there as regression tests.The smaller class, such as
'2405 Uurku B986445-9 Ni Pa (Ashdak Meshukiiba) Sa { 0 } (A33-2) [2437] Bc - - 222 8 ImDv K1 V'are things I tripped over when running the parsing part of a Charted Space run, that also violated that test's specification. Likewise, I've left them there as regression tests.