melbahja / got

Got: Simple golang package and CLI tool to download large files faster 🏃 than cURL and Wget!
MIT License
709 stars 45 forks source link

About file size #63

Open amandaluof opened 1 year ago

amandaluof commented 1 year ago

Hi, thanks for your great project, it is really great !!

But I met a problem when using your tool to download ImageNet-21k(https://image-net.org/data/winter21_whole.tar.gz). It is a single file and its size is about 1.2T.

After downoding 411GB of the file, it stucks with erorr saying "file too large". Could you please tell me how to avoid the mistake?

Thank you very much!

melbahja commented 1 year ago

In case of huge files like that I think the ideal solution is by using retries and resume downloads