search

nationalparkservice / NPSutils

NPSutils contains functions for accessing, downloading, loading in to R and filtering data packages on DataStore
https://nationalparkservice.github.io/NPSutils/
Other
6 stars 2 forks source link

Failed downloads create CSV files that contain HTML errors #56

Closed juddpatterson closed 1 month ago

juddpatterson commented 1 month ago

Downloading files that exist, but fail (for various reasons) can result in CSV files that only contain HTML. The filenames are correct (e.g. MIDN_NCBN_NETN_Landbirds_20240909.csv) but the actual data is missing. Both the presence of files on the file system and the feedback of the function itself make it appear that get_data_packages() has succeeded. The missing data is discovered only once you try to look inside the data frames that are loaded.

Here is one example that was failing and doing this for a user:

NPSutils::get_data_packages(2305911, secure = TRUE) #MIDN, NETN, NCBN landbirds, draft'
dat <- NPSutils::load_data_package(2305911) 
list2env(dat, envir = .GlobalEnv)
RobLBaker commented 1 month ago

I tried this and I'm getting the following error:

Reference 2305911 does not exist. Are you logged on to the VPN?

I also can't find that reference on DataStore using the web interface. I suspect the problem is that is a restricted reference that I don't have access to as I am on the VPN and it appears that DataStore has validated my credentials/I am logged in.

Is this happening to other references and/or can I get access to this reference?

juddpatterson commented 1 month ago

This is happening on others, but I'd have to dig in a little on those other codes. You've been added as an owner on reference 2305911. Thanks for taking a look!

RobLBaker commented 1 month ago

Further investigation reveals that this is an error on the DataStore side; it is (was) serving up a bad downloadLink URL. That has been fixed now, so closing this issue.