Where is the right place to store test data files for testing Web Neural Network API?

[BruceDai]

According to discussion on WebML CG meeting, we will test Web Neural Network API by both op level and model level test cases. Now I'm starting to add some op level test cases and model level test cases into WPT. Regarding to model level test cases, since the test data of model's weights, input, baseline are much larger, we store them into test data files which are stored in webmachinelearning/test-data repository, and current the total size of those test data files is 651M, and it would be larger later.

I'm not sure whether it's OK to add those test data files into WPT repository. Or, where is the right place to store them?

Any suggestion @dontcallmedom @anssiko @Honry, thanks.

[anssiko]

webmachinelearning/test-data is almost the size of the current web-platform-tests/wpt repo, so perhaps test-data could be used as a git submodule in wpt?

@dontcallmedom suggestions?

[dontcallmedom]

before figuring the how (where we will need to consult the WPT project owners), let's figure out whether we actually need 615M of data to properly test WebNN.

My sense is that https://github.com/webmachinelearning/test-data is probably a more thorough test of models than would be appropriate in WPT. Are all the models needed to achieve our interoperability goals? what have been the selection criteria for adding test models in that repo? What is overall our strategy around testing WebNN on models?

[anssiko]

These models map to WebNN use cases: https://www.w3.org/TR/webnn/#usecases

Different models exercise different ops: https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md

Regardless of the specifics, I think we want to find a solution that does not grow the wpt repo size 2x or even 1.5x.

@wchao1115 for thoughts on ”Are all the models needed to achieve our interoperability goals?”