search

HenrikBengtsson / R.utils

🔧 R package: R.utils (this is *not* the utils package that comes with R itself)
https://henrikbengtsson.github.io/R.utils/
62 stars 14 forks source link

`downloadFile` doesn't honor option("timeout") when binary is found #146

Open xbello opened 1 year ago

xbello commented 1 year ago

I'm trying to download some huge files (about 3Gb), that takes longer than 120 seconds to download. The library is GEOquery, which uses R.utils::downloadFile.

When the schema to download is https, the function first checks if curl or wget binaries can be found in the system. Lets suppose curl is found, next steps is to add args to the command line, such as --insecure, --location, --user, and then proceed to download through system2 at https://github.com/HenrikBengtsson/R.utils/blob/b53c2291fbd41b2bc66991c8586c1af1fabbd9ff/R/downloadFile.R#L213

Usually you can set a timeout option with option("timeout"=N), and download.file honours that setting, but both curl and wget doesn't. I barely speak R-lang, but my suggestions would be to either respect the timeout setting, or allow passing extra args to curl/wget (--connect-timeout and --timeout, respectively) that could be used to pre-load args at https://github.com/HenrikBengtsson/R.utils/blob/b53c2291fbd41b2bc66991c8586c1af1fabbd9ff/R/downloadFile.R#L162 instead of using NULL.