psychoinformatics-de / datalad-hirni

DataLad extension for (semi-)automated, reproducible processing of (medical/neuro)imaging data
http://datalad.org
Other
5 stars 8 forks source link

Prepare release 0.0.5 #153

Closed bpoldrack closed 4 years ago

bpoldrack commented 4 years ago

This release should most importantly lead to compatibility with datalad 0.12.0rc6+, the lack thereof shown particularly in #129 and #150. Fix for that is in master already, release needs to make sure it's actually available, get dependencies right and include some minor fixes while at it.

bpoldrack commented 4 years ago

Remaining failure in two builds:

======================================================================

ERROR: datalad_hirni.tests.test_demos.test_demo_repro_analysis

----------------------------------------------------------------------

Traceback (most recent call last):

  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/nose/case.py", line 198, in runTest

    self.test(*self.arg)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/tests/utils.py", line 156, in newfunc

    return func(*args, **kwargs)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/tests/utils.py", line 727, in newfunc

    return t(*(arg + (filename,)), **kw)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/tests/utils.py", line 727, in newfunc

    return t(*(arg + (filename,)), **kw)

  File "/home/travis/build/psychoinformatics-de/datalad-hirni/datalad_hirni/tests/test_demos.py", line 172, in test_demo_repro_analysis

    analysis_ds.containers_add('fsl', url="shub://ReproNim/ohbm2018-training:fsln")

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/distribution/dataset.py", line 526, in apply_func

    return f(**kwargs)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/interface/utils.py", line 493, in eval_func

    return return_func(generator_func)(*args, **kwargs)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/interface/utils.py", line 481, in return_func

    results = list(results)

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/interface/utils.py", line 412, in generator_func

    allkwargs):

  File "/home/travis/virtualenv/python3.5.6/src/datalad/datalad/interface/utils.py", line 528, in _process_results

    for res in results:

  File "/home/travis/virtualenv/python3.5.6/src/datalad-container/datalad_container/containers_add.py", line 223, in __call__

    imgurl = _resolve_img_url(url)

  File "/home/travis/virtualenv/python3.5.6/src/datalad-container/datalad_container/containers_add.py", line 42, in _resolve_img_url

    shub_info = loads(req.text)

  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/simplejson/__init__.py", line 525, in loads

    return _default_decoder.decode(s)

  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/simplejson/decoder.py", line 370, in decode

    obj, end = self.raw_decode(s)

  File "/home/travis/virtualenv/python3.5.6/lib/python3.5/site-packages/simplejson/decoder.py", line 400, in raw_decode

    return self.scan_once(s, idx=_w(s, idx).end())

simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This appears to be an issue with loading JSON response from Singularity Hub. (containers_add._resolve_img_url("shub://ReproNim/ohbm2018-training:fsln"))

However, I cannot reproduce that locally and fail to see where my local env is different. Any chance that failure rings a bell, @kyleam?

kyleam commented 4 years ago

Any chance that failure rings a bell, @kyleam?

No, sorry. My wild guess is that it's due to Singularity Hub's download limits (in which case containers-add should of course give a more informative error).

bpoldrack commented 4 years ago

Thx, @kyleam ! Your guess seems not to far off. Tried again to reproduce from within datalad-containers tests only (by removing the skips due to shub "being down"). Got the same thing (additional logging capture says we got a 502) on first try, but can't reproduce again.

bpoldrack commented 4 years ago

And finally I made into debugger. It's 403 You've done that too many times, try again tomorrow.

codecov-io commented 4 years ago

Codecov Report

Merging #153 into master will decrease coverage by 0.09%. The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #153     +/-   ##
=========================================
- Coverage   86.27%   86.17%   -0.1%     
=========================================
  Files          17       17             
  Lines         947      955      +8     
=========================================
+ Hits          817      823      +6     
- Misses        130      132      +2
Impacted Files Coverage Δ
datalad_hirni/tests/test_dicom2spec.py 100% <100%> (ø) :arrow_up:
datalad_hirni/commands/dicom2spec.py 84.27% <100%> (+2.22%) :arrow_up:
datalad_hirni/tests/test_demos.py 92.85% <100%> (-7.15%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 67d546b...d1de877. Read the comment docs.