search

cioos-siooc / ckan

CKAN is an open-source DMS (data management system) for powering data hubs and data portals. CKAN makes it easy to publish, share and use data. It powers datahub.io, catalog.data.gov and europeandataportal.eu/data/en/dataset among many other sites.
http://ckan.org/
Other
2 stars 4 forks source link

If modified date is not found in the waf, the harvesting process fails #134

Closed fostermh closed 2 years ago

fostermh commented 2 years ago

Reported by @ogsletcax .

When no modified date is found in the waf, the harvesting process fails. It should not.

Notes on current behaviour: when waf is served by apache it is expected that the options +FancyIndexing +HTMLTable are set.

when waf is served by apache but behind an nginx proxy, it will fail to find the modified date because the request header “server’ will contain “nginx” instead of “apache”

fostermh commented 2 years ago

need an example waf to test solution.

ogsletcax commented 2 years ago

An example of apache file listing with FancyIndexing and HTMLTable options behind an nginx proxy: https://catalogue-waf.preprod.ogsl.ca/metadata-files/merinov/

An example of apache file listing without formating options behind an nginx proxy: https://catalogue-waf.ogsl.ca/metadata-files/mpo/

fostermh commented 2 years ago

closing this for now as it appears to work with the existing harvest setup. If more problems are found we can re-open.