Closed RMHogervorst closed 6 years ago
I recently wanted to download hundreds of files However I had to wait for a long time, didn't know how far I was (could have checked the folder)
A solution to help me, because I'm easily bored, get feedback is to wrap a messaging function around a function that you call many times.
#' download a file and give feedback download_file <- function(file){ filename <- basename(file) if(file.exists(paste0("data/",filename))){ print(paste("file exists: ",filename)) }else{ print(paste0("downloading file:", file)) curl::curl_download(url = file,destfile = paste0("data/",filename),mode = "wb") Sys.sleep(0.3) } }
I then apply the function to a list of adresses:
map(paste0("first part of the link", formatC(1:latest_episode, width = 3,flag = 0),".txt"), download_file)
According to this SO post the formatc function is much faster then other options.
I recently wanted to download hundreds of files However I had to wait for a long time, didn't know how far I was (could have checked the folder)
A solution to help me, because I'm easily bored, get feedback is to wrap a messaging function around a function that you call many times.
I then apply the function to a list of adresses: