bleonar5
commented
10 months ago Fortunately, the BIDS validator has already included a lot of unit tests in their deno app, I just hadn't included them yet since I was getting the initial version off the ground. I'll make a new branch and add these in, plus I'll fill in the gaps of what they haven't added yet, since it seems like their testing implementation is incomplete.
To clarify on your confusion, I think we should separate out the ideas of unit testing and dataset testing. For the purposes of unit testing, we want to have test versions of every function included in our application, which we can use to confirm that they don't break when we give them standard test inputs. For dataset testing, we want to loop through our list of example datasets, run the full validator on them, and confirm that they produce the result we expect.
At the appropriate moment (soon?), start and/or document a unit testing framework; see BIDS and HED for models.
Note: MKS is not clear whether these unit tests include, or are separate from, the validation over the reference datasets at https://github.com/bids-standard/bids-examples and https://github.com/hed-standard/hed-examples. Leaving as a single issue for now.
(Note also: https://github.com/psych-ds/example-datasets currently contains contributed datasets that do not meet the spec, and this is an expected state at this point. In the future, will need to establish pathways for how people can contribute dataset examples (a) generally and (b) for unit testing.)