neuro-inc / ml-recipe-bone-age

Pediatric Bone Age Assessment
Apache License 2.0
25 stars 6 forks source link

Fix data management + Add jupyter tests #6

Closed atemate closed 4 years ago

atemate commented 4 years ago

Use public data storage:/neuromation/public/bone-age

atemate commented 4 years ago

Note: unzipping archive http://data.neu.ro/bone-age.zip exits with 2:

Exit code: 2
===Description===
.......... .......... .......... 99%  182M 0s
7656150K .......... .......... .......... .......... .......... 99% 87.6M 0s
7656200K .......... .......... .......... .......... .......... 99%  219M 0s
7656250K .......... .......... .......... .......... .......... 99%  226M 0s
7656300K .......... .......... .......... .......... .......... 99% 71.4M 0s
7656350K .......... .......... .......... .......... .......... 99%  142M 0s
7656400K .......... .......... .......... .......... .......... 99%  206M 0s
7656450K .......... .......... .......... .......... .......... 99%  107M 0s
7656500K .......... .......... .......... .......... .......... 99%  181M 0s
7656550K .......... .......... .......... .......... .......... 99%  119M 0s
7656600K .......... .......... .......... .......... .......... 99% 3.08M 0s
7656650K .......... .......... .......... .......... .......... 99% 48.3M 0s
7656700K .......... .......... .......... .......... .......... 99% 44.1M 0s
7656750K .......... .......... .......... .......... .......... 99%  145M 0s
7656800K .......... .......... .......... .......... .......... 99% 74.7M 0s
7656850K .......... .......... .......... .......... .......... 99%  180M 0s
7656900K .......... .......... .......... .......... .......... 99%  211M 0s
7656950K .......... .......... .......... .......... .......... 99%  130M 0s
7657000K .......... .......... .......... .......... .......... 99%  153M 0s
7657050K .........                                             100%  178M=3m8s

2020-04-07 13:17:45 (39.8 MB/s) - ‘/tmp/bone-age.zip’ saved [7840828536/7840828536]

Archive:  /tmp/bone-age.zip
warning [/tmp/bone-age.zip]:  3408 extra bytes at beginning or within zipfile
  (attempting to process anyway)
error [/tmp/bone-age.zip]:  reported length of central directory is
  -3408 bytes too long (Atari STZip zipfile?  J.H.Holm ZIPSPLIT 1.1
  zipfile?).  Compensating...
replace /data/models/bone_age.epoch36-err0.059.pth? [y]es, [n]o, [A]ll, [N]one, [r]ename:  NULL
(EOF or read error, treating as "[N]one" ...)

Full command:

neuro run -s cpu-small -v storage:public/ml-recipe-bone-age/:/data:rw -n download-bone-age neuromation/base:latest bash -c 'wget http://data.neu.ro/bone-age.zip -O /tmp/bone-age.zip && unzip /tmp/bone-age.zip -d /data'