show dl progress, dl location, honor md5check

lawinslow / hydrolinks

National Hydrography Network Dataset Tools for R

Other

10 stars 4 forks source link

show dl progress, dl location, honor md5check #41

Closed jsta closed 6 years ago

jsta commented 6 years ago

I was having trouble downloading flowtable_nhdplusv2.zip in the process of running the traverse_flowlines example. I kept getting the error:

Error in FUN(newX[, i], ...) : Data downloaded was incomplete or corrupted - please try again later or submit a bug report.

This PR:

adds a progress bar to file downloads
prints the URL of the target file
modifies check_dl_file to only compare md5sum if md5check is TRUE

lawinslow commented 6 years ago

Hmm, did you add the ... pass-through because you are trying to disable the MD5 check on downloads? Or do you have another top use-case? Disabling the MD5 check on initial download would likely just cause issues down the road with unexpected or corrupt data.

jsta commented 6 years ago

Yes. That was the only way I could get traverse_flowlines to run was by disabling the MD5 check. Otherwise I get the error above. Maybe the MD5 check conflict is only on my end? Does the reference table have the correct value?

lawinslow commented 6 years ago

@rainshapes is looking into it. That may be an issue on our end. The data processing workflow still has some kinks to work out.

I'm going to merge this (Thanks by the way!), but if we can more solidly workout our data workflow in the future, I may switch back to having the MD5 absolutely required.