DECALS download freezes at 96%

zooniverse / decals

Morphological classifications for the Galaxy Zoo-DECaLS collaboration

MIT License

7 stars 3 forks source link

DECALS download freezes at 96% #16

Closed mwalmsley closed 6 years ago

mwalmsley commented 6 years ago

Stops at 96% consistently…

96%|█████████▋| 333323/346044 [4:55:50<1:30:20, 2.35 images created/s

Compare with another run cut to 333300 images

mwalmsley commented 6 years ago

This has been resolved by crying for weeks and then realising urllib (and by extension requests) don't support multithreading happily, causing threads to die silently and the pool to hang. Instead, wget is used via command line interface.