metno / discovery-metadata-catalog-ingestor

Apache License 2.0
1 stars 1 forks source link

Not able to batch update MMD files because dataset not found #207

Closed mortenwh closed 11 months ago

mortenwh commented 11 months ago

Some datasets cannot be updated easily, since they cannot be found through the CSW API. For example no.met.dev:e7bc9358-8cb5-49d4-8024-76846a366838:

It exists in git: https://gitlab.met.no/mmd/mmd-xml-dev/-/blob/master/arch_8/arch_5/arch_3/e7bc9358-8cb5-49d4-8024-76846a366838.xml?ref_type=heads

Does it exist in pycsw?

At least it is not found:

curl "https://csw.s-enda-dev.k8s.met.no/csw?service=CSW&version=2.0.2&request=GetRecordById&oxml&outputSchema=http://www.isotc211.org/2005/gmd&elementsetname=full&id=no.met.dev:e7bc9358-8cb5-49d4-8024-76846a366838"
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.7.dev0 -->
<csw:GetRecordByIdResponse xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/cat/csw/2.0.2 http://schemas.opengis.net/csw/2.0.2/CSW-discovery.xsd"/>

But it can't be added either:

curl --data-binary "@arch_8/arch_5/arch_3/e7bc9358-8cb5-49d4-8024-76846a366838.xml" https://dmci.s-enda-dev.k8s.met.no/v1/insert
The following distributors failed: file, pycsw, solr
 - file: File already exists: e7bc9358-8cb5-49d4-8024-76846a366838.xml
 - pycsw: <?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- pycsw 2.7.dev0 -->
<ows:ExceptionReport xmlns:csw="http://www.opengis.net/cat/csw/2.0.2" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:dct="http://purl.org/dc/terms/" xmlns:gmd="http://www.isotc211.org/2005/gmd" xmlns:gml="http://www.opengis.net/gml" xmlns:ows="http://www.opengis.net/ows" xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.2.0" language="en-US" xsi:schemaLocation="http://www.opengis.net/ows http://schemas.opengis.net/ows/1.0.0/owsExceptionReport.xsd"><ows:Exception exceptionCode="NoApplicableCode" locator="insert">
<ows:ExceptionText>Transaction (insert) failed: (psycopg2.errors.ReadOnlySqlTransaction) cannot execute INSERT in a read-only transaction

...
(Background on this error at: http://sqlalche.me/e/13/2j85).</ows:ExceptionText></ows:Exception></ows:ExceptionReport>
 - solr: Document already exists in index, no.met.dev:e7bc9358-8cb5-49d4-8024-76846a366838

Not sure what the problem is but maybe a rebuild is enough? @magnarem - any thoughts?