search

astropy / pytest-remotedata

Pytest plugin to control whether tests are run that have remote data
BSD 3-Clause "New" or "Revised" License
23 stars 15 forks source link

Configure strict remote data checking #20

Closed drdavella closed 7 years ago

drdavella commented 7 years ago

This fixes #18 by allowing strict remote data checking to be configurable. This means that tests that access remote data sources but are not marked with @pytest.mark.remote_data will only fail if the configuration option remote_data_strict has been set in setup.cfg.

Astropy and affiliated packages that want this behavior will have to add remote_data_strict = true to their setup.cfg files.

pllim commented 7 years ago

I was just wondering if strict should be default, but rather have an option to opt out. So in that case, astropy don't need to add extra config, but JWST pipeline does.

pllim commented 7 years ago

And in the case of JWST pipeline, where its usage is actually defined within asdf dependency, how would this feature affect the dynamics over there? That is, would opting out have unintended consequences for the asdf dependency part?

drdavella commented 7 years ago

@pllim no I think the default needs to be disabled. Otherwise this plugin could potentially cause failures for lots of other packages that care nothing about remote data tests.

drdavella commented 7 years ago

And wrt asdf/jwst, the option is configured by the package being tested, so even if a dependency has this option set in it's own setup.cfg, it should not affect the package being tested if it does not.

pllim commented 7 years ago

@drdavella , that is a good point. But for packages that already using remote_data, now those marked tests won't be skipped anymore unless they add this "strict" config, is that right?

bsipocz commented 7 years ago

I was also thinking about packages that already use remote-data. Is that right that they cannot switch to use it from this package while they keep up support for astropy <3.0, as the plugins there and here are conflicting?

drdavella commented 7 years ago

@bsipocz, yes that is true right now. I have started thinking about whether there's something we can backport to v2.0.x to make it work, but I'm not sure whether it will be possible.

drdavella commented 7 years ago

@pllim, no this option only affects whether tests that access the internet but are not marked with @pytest.mark.remote_data fail. The rest of the remote_data functionality should work as expected.

bsipocz commented 7 years ago

Well, if we could do some backporting it would be brilliant.