Lightning-Universe / lightning-flash

Your PyTorch AI Factory - Flash enables you to easily configure and run complex AI recipes for over 15 tasks across 7 data domains
https://lightning-flash.readthedocs.io
Apache License 2.0
1.74k stars 212 forks source link

`download_data` from `flash.core.data.utils` connects to the internet before checking if a file exists #1611

Closed surak closed 1 year ago

surak commented 1 year ago

🐛 Bug

In many supercomputers, process of running ML codes is to first run the download part on the login nodes (which have access to the internet), and stop the code right before the actual training starts.

Then, when you run on the compute nodes (the ones with the actual gpus and no internet access), you let the code run to the end. In other frameworks, data downloaders detect the files' presence and skip it before ever trying to connect to the internet.

Flash tries first to check file size in this line, which will freeze in a machine without internet.

To Reproduce

call "download_data" on a machine with no internet access

Code sample

from flash.core.data.utils import download_data
download_data("https://pl-flash-data.s3.amazonaws.com/hymenoptera_data.zip", "data/")

Expected behavior

If the file is already there, skip download

Environment

Additional context

Fast.ai's fastdownload, for example, does not suffer from this - if the file is there, it doesn't try to download it, even if it's of the wrong size: fastdownload link

Borda commented 1 year ago

Nice catch. :detective: Would you be interested in sending a fix? :rabbit: