search

Unidata / siphon

Siphon - A collection of Python utilities for retrieving atmospheric and oceanic data from remote sources, focusing on being able to retrieve data from Unidata data technologies, such as the THREDDS data server.
https://unidata.github.io/siphon
BSD 3-Clause "New" or "Revised" License
213 stars 75 forks source link

Handle HTTP redirects properly #53

Open lesserwhirls opened 9 years ago

lesserwhirls commented 9 years ago

To be consistent with other services, TDS 5.0 serves out all catalogs using

thredds/catalog

The top level catalog's old url:

thredds/catalog.(html|xml)

redirects to:

thredds/catalog/catalog.(html|xml)

which siphon catches. However, the TDSCatalog object still has the old url, and thus any relative catalogRefs are constructed incorrectly (except for catalogRefs generated by catalogScan elements). For example, a TDS 5.0 catalogRef to forecastModels.xml is turned into an absolute URL of:

/thredds/forecastModels.xml instead of the new, correct absolute URL of: /thredds/catalog/forecastModels.xml A request to the old absolute URL does not get a redirect, and thus fails with a 404. Perhaps the correct thing to do here is check for the redirect to set the catalog objects "cat_url" attribute in TDSCatalog and make sure that absolute URLs for catalogRefs are built with the correct url. This should keep backwards compatibility with TDS <= 4.6 while future proofing for TDS 5.0.
lesserwhirls commented 6 years ago

The real issue is that we do not handle http redirects properly.