--remote-data=astropy does not work properly?

[pllim]

I am testing something in Astropy v2.0.x branch in Python 2 environment and I noticed that --remote-data=astropy and --remote-data=none skip the same tests even when I know there are some tests marked as @remote_data('astropy') in the test file. I was expecting that --remote-data=astropy would skip less tests (i.e., run more tests).

$ python setup.py test -t astropy/utils/tests/test_data.py --remote-data=astropy
============================= test session starts ==============================
platform linux2 -- Python 2.7.14, pytest-3.5.0, py-1.5.3, pluggy-0.6.0

Running tests with Astropy version 2.0.7.dev19589.
Running tests in astropy/utils/tests/test_data.py.

Date: 2018-05-23T15:30:08

Platform: Linux-3.10.0-862.el7.x86_64-x86_64-with-redhat-7.5-Maipo

Executable: .../anaconda/envs/iraf27/bin/python

Full Python Version: 
2.7.14 |Anaconda, Inc.| (default, Mar 27 2018, 17:29:31) 
[GCC 7.2.0]

encodings: sys: ascii, locale: UTF-8, filesystem: UTF-8, unicode bits: 20
byteorder: little
float info: dig: 15, mant_dig: 15

Numpy: 1.13.3
Scipy: 1.0.1
Matplotlib: 2.2.2
h5py: 2.7.1
Pandas: 0.22.0
Cython: 0.28.1
Using Astropy options: remote_data: astropy.

rootdir: /tmp/astropy-test-2PR7lr/lib/python2.7/site-packages, inifile: setup.cfg
plugins: remotedata-0.2.1, openfiles-0.3.0, doctestplus-0.1.3, arraydiff-0.2
collected 58 items                                                             

astropy/utils/tests/test_data.py .sssssss..........s...x.s.s.s.sssssss.. [ 50%]
........s...x.s.s.s
<snipped>
========= 34 passed, 22 skipped, 2 xfailed, 2 warnings in 0.38 seconds =========

$ python setup.py test -t astropy/utils/tests/test_data.py
<snipped>
========= 34 passed, 22 skipped, 2 xfailed, 2 warnings in 0.39 seconds =========

But when I run with --remote-data (any), they are not skipped anymore.

Here is an example test function in the given test file:

@remote_data('astropy')
def test_download_parallel():
    ...

[pllim]

I also see this in the pytest warnings. Now I am just very confused.

Module already imported so cannot be rewritten: pytest_remotedata

[pllim]

I am thinking this is a real bug and it is biting me because there is a failed test that is not caught in https://travis-ci.org/astropy/astropy/jobs/382189395#L4542 , a Travis job for Python 2 that is testing --remote-data=astropy.

[pllim]

And looks like it is still happening in master as well, not just v2.0.x.

This is using --remote-data=astropy: https://travis-ci.org/astropy/astropy/jobs/382829806#L5025

astropy/utils/tests/test_data.py sssssss...........s.....s.s..           [ 93%]

This is using just --remote-data: https://travis-ci.org/astropy/astropy/jobs/382829810#L5028

astropy/utils/tests/test_data.py .............................           [ 93%]

[pllim]

Maybe it is because of this logic:

        elif remote_data_config == 'astropy':
            if source == 'any':
                pytest.skip("need --remote-data option to run")

When --remote-data=astropy is given, remote_data_config == 'astropy' is True and source == 'any' is also True by default. I can't say I understand the basis of that logic block.

[drdavella]

Just to record for posterity: the problem here is actually that the tests in question do not use the source= kwarg when passing the source argument to the remote_data decorator. On the one hand, we can support this alternative calling scheme (which is what #27 provides). On the other hand, we could just make Astropy fix these "degenerate" cases.