do not exclude travis markers in plugin test runner

brain-score / core

MIT License

1 stars 6 forks source link

do not exclude travis markers in plugin test runner #15

Open mschrimpf opened 2 years ago

mschrimpf commented 2 years ago

the plugin runner currently excludes all tests that cannot be run on travis: https://github.com/brain-score/language/blob/dac042286911a0f42d897982fcf9ac45d3775c18/brainscore_language/plugin_management/test_plugin.sh#L25

This exclusion should only be set for travis runs (e.g. in .travis.yml) so that such tests in general are not excluded, e.g. for execution on openmind.

kvfairchild commented 2 years ago

Should we not exclude any tests as a default? Or should we have some default exclusions (e.g. not requires_gpu) and override as needed?

mschrimpf commented 2 years ago

I think the running system should set these exclusions and ideally be as non-restrictive as possible.