SignalK / specification

Signal K is a JSON-based format for storing and sharing marine data from different sources (e.g. nmea 0183, 2000, seatalk, etc)
Other
91 stars 69 forks source link

refactor: improve usability of json validation tests #422

Closed bkp7 closed 6 years ago

bkp7 commented 6 years ago

This PR refactors the test layout and way tests are called but does not change any tests, test methodology or test results.

Previously most tests consisted of a file and a specific piece of javascript to run the test on that file.

Under this revision there is generic javascript which runs tests on any files it finds in the test/data subfolders based on which subfolder it is in. Going forwards this means that simply placing a file into one of the folders will include it in tests.

There are some places where the test json was coded into the javascript. In these cases a new file containing identical json has been created and placed in the appropriate folder.

There were some files which were being tested against 'full without self', and in these cases 'self' has been added to the file so they can be tested against the full schema.

Before this PR there were 119 tests all of which reported as passed. Following these changes that has increased to 137 due to:

Currently the test results are not laid out particularly logically, but they weren't previously anyway. I have adopted a file naming convention for tests as follows: [test area]-[test description].json. Note that there is no need to mention in the filename the schema being tested against or whether the json is valid/invalid as this is known from the folder it is in.

bkp7 commented 6 years ago

Any chance of committing this or having some feedback please?

There are a number of issues outstanding which require changes/fixes to the schema and it would be much better to add new tests this way to ensure those changes are made correctly, backed up by comprehensive test json. Under the existing regime each test has to be hand crafted, which is prone to error and a deterrent to producing full coverage tests.

Thanks.

tkurki commented 6 years ago

This is great stuff - thank you very much!