datalad / datalad-ukbiobank

Resources for working with UKBiobank as a DataLad dataset
MIT License
6 stars 12 forks source link

Test failure `test_drop` #92

Closed mih closed 2 years ago

mih commented 2 years ago
===================================
__________________________________ test_drop ___________________________________

dspath = '/tmp/datalad_temp_test_drop3madglh0'
records = '/tmp/datalad_temp_test_drop8s8mzg2j'

    @skip_if_on_windows  # see gh-61
    @with_tempfile
    @with_tempfile(mkdir=True)
    def test_drop(dspath=None, records=None):
        make_datarecord_zips('12345', records)
        ds = create(dspath, **ckwa)
        ds.ukb_init(
            '12345',
            ['20227_2_0', '25747_2_0', '25748_2_0', '25748_3_0'], **ckwa)
        ds.config.add('datalad.ukbiobank.keyfile', 'dummy', where='local')
        bin_dir = make_ukbfetch(ds, records)

        # baseline
        with patch.dict('os.environ', {'PATH': '{}:{}'.format(
                str(bin_dir),
                os.environ['PATH'])}):
            ds.ukb_update(merge=True, force=True, **ckwa)
        zips_in_ds = list(ds.pathobj.glob('**/*.zip'))
        neq_(zips_in_ds, [])

        # drop archives
        with patch.dict('os.environ', {'PATH': '{}:{}'.format(
                str(bin_dir),
                os.environ['PATH'])}):
            ds.ukb_update(merge=True, force=True, drop='archives', **ckwa)
        # no ZIPs can be found, also not in the annex
        eq_(list(ds.pathobj.glob('**/*.zip')), [])
        # we can get all we want (or rather still have it)
>       assert_status('notneeded', ds.get('.', **ckwa))

/opt/hostedtoolcache/Python/3.7.15/x64/lib/python3.7/site-packages/datalad_ukbiobank/tests/test_update.py:212: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

label = ['notneeded'], results = []

    def assert_status(label, results):
        """Verify that each status dict in the results has a given status label

        `label` can be a sequence, in which case status must be one of the items
        in this sequence.
        """
        label = ensure_list(label)
        results = ensure_result_list(results)
        if len(results) == 0:
            # If there are no results, an assertion about all results must fail.
>           raise AssertionError("No results retrieved")
E           AssertionError: No results retrieved
mih commented 2 years ago

I looked into this now, and the test failure is merely a signaling (change?) in datalad get. The functionality that is tested here (drop) works as expected.

What is happening is that datalad get . does not yield any results whenever nothing needs to be downloaded (in contrast to datalad get *).

The relevant get call can simply be removed from the test. The line above already verifies that all ZIP archives where dropped. And the following block verifies that re-extracting from re-downlaoded ZIPs also works.