search

Kaggle / kaggle-api

Official Kaggle API
Apache License 2.0
6.28k stars 1.1k forks source link

Support for resumed downloads (enabled by default) #462

Closed csachs closed 1 year ago

csachs commented 1 year ago

Reopen of #328 (can't change the target branch). ( Should fix #24 , #310 , #316 )

DapengFeng commented 1 year ago

Thanks for your work. But I am still confused about how to resume the download process when failing in the following command

kaggle competitions download -c imagenet-object-localization-challenge
tmijieux commented 1 year ago

Thanks for your work. But I am still confused about how to resume the download process when failing in the following command

kaggle competitions download -c imagenet-object-localization-challenge

I did not test it yet, but seeing the code i would say that you just have to relaunch the same command, it automatically detects if the file exists and will send appropriate headers according to existing file size to start again where it stopped.

EDIT: since the check for existing file is done just before the modified function is called , you need to add the --force option

tmijieux commented 1 year ago

@csachs maybe you could also change the return of the download_needed to True when

exactly like what you did at the beginning of the download_file function

that way it would avoid to have to pass the --force option.

Also i believe the --force is meant to force the download to restart maybe that point could be discussed, but IF my belief about this option behavior is right, an idea would be to pass not force to the new resume parameter if the user truly want to restart rather than continue an existing file (i dont really why someone would want to do that unless they have some way to know that the part they already downloaded is corrupted or bad in some way)

Philmod commented 1 year ago

Thanks @csachs and @tmijieux, I'm going to implement this solution.

The source of code is internal, and is copied here, that's why I cannot merge this PR directly.

Philmod commented 1 year ago

https://github.com/Kaggle/kaggle-api/releases/tag/1.5.14