search

GEUS-Glaciology-and-Climate / pypromice

Process AWS data from L0 (raw logger) through Lx (end user)
https://pypromice.readthedocs.io
GNU General Public License v2.0
14 stars 4 forks source link

Data URL updates #111

Closed PennyHow closed 1 year ago

PennyHow commented 1 year ago

I'm still struggling with updating the data URLs used in pypromice.get as there is a bit of a lag between a Dataverse dataset release and the DOIs updating at src/pypromice/data_urls.csv.

So I thought linking to the thredds server links was a better solution, as it unlikely these URLs are going to change frequently. I don't know if this is suitable, but it guarantees that the user gets the latest data and not old data through a stale link.

Let me know what you think!

patrickjwright commented 1 year ago

Yes I think this works fine, particularly if having these URLs point to NRT data is a priority.

PennyHow commented 1 year ago

Given that we want to point users to the Dataverse products, I am going to find a way to fetch the latest data file DOI from the Dataverse rather than using the thredds NRT data links.

So I will close this PR for now and make a new one once I have the Dataverse DOI fetching in place.

Maybe later down the line we can link directly to the thredds NRT data links. That will be an exciting time!