Connect CIOOS (national) catalogue into ODIS

iodepo / odis-arch

Development of the Ocean Data and Information System (ODIS) architecture

https://book.oceaninfohub.org/

26 stars 16 forks source link

Connect CIOOS (national) catalogue into ODIS #321

Open jmckenna opened 10 months ago

jmckenna commented 10 months ago

related to initial issue https://github.com/iodepo/odis-arch/issues/84

@fostermh

jmckenna commented 10 months ago

related: new repo location for the JSON-LD "framing" script: https://github.com/iodepo/odis-arch/blob/master/utils/jsonldProcessing/framing/framing-json-ld.py

jmckenna commented 1 month ago

issue: the CIOOS robots.txt sets a Crawl-Delay of 10 seconds, which causes extra processing time for ODIS - we can override this, but 10 seconds seems excessive. Wondering if the CIOOS team can change that to something like 1 or 2 seconds instead.