search

collectionspace / collectionspace-mapper

MIT License
0 stars 0 forks source link

truncated XML result returned (when no API response?) #105

Open kspurgin opened 3 years ago

kspurgin commented 3 years ago

Between 19:10 and 19:26 EST on 2021-04-14, I ran a batch of >1,000 rows through cspace-batch-import to create and transfer fcart Acquisition records into ba-staging.

All the acquisition records were created and transferred as expected.

However...

This was the XML for one of the rows transferred by cspace-batch-import during that time frame: 

<?xml version="1.0"?>
<document>
    <ns2:acquisitions_common xmlns:ns2="http://collectionspace.org/services/acquisition"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <acquisitionReferenceNumber>UU1881.1.1 (migrated accession)</acquisitionReferenceNumber>
        <acquisitionMethod>purchase</acquisitionMethod>
    </ns2:acquisitions_common>
</document>

This is the XML for the same row, transferred by cspace-batch-import this morning:

<?xml version="1.0"?>
<document>
    <ns2:acquisitions_common xmlns:ns2="http://collectionspace.org/services/acquisition"
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <acquisitionReferenceNumber>UU1881.1.1 (migrated accession)</acquisitionReferenceNumber>
        <accessionDateGroup>
            <dateDisplayDate>1881</dateDisplayDate>
            <dateEarliestSingleYear>1881</dateEarliestSingleYear>
            <dateEarliestSingleMonth>1</dateEarliestSingleMonth>
            <dateEarliestSingleDay>1</dateEarliestSingleDay>
         <dateEarliestSingleEra>urn:cspace:ba.collectionspace.org:vocabularies:name(dateera):item:name(ce)'CE'</dateEarliestSingleEra>
            <dateLatestYear>1881</dateLatestYear>
            <dateLatestMonth>12</dateLatestMonth>
            <dateLatestDay>31</dateLatestDay>             
           <dateLatestEra>urn:cspace:ba.collectionspace.org:vocabularies:name(dateera):item:name(ce)'CE'</dateLatestEra>
            <dateEarliestScalarValue>1881-01-01T00:00:00.000Z</dateEarliestScalarValue>
            <dateLatestScalarValue>1882-01-01T00:00:00.000Z</dateLatestScalarValue>
            <scalarValuesComputed>true</scalarValuesComputed>
        </accessionDateGroup>        <acquisitionAuthorizer>urn:cspace:ba.collectionspace.org:personauthorities:name(person):item:name(CharlesCallahanPerkins182318861213560177)'Charles Callahan Perkins (1823-1886)'</acquisitionAuthorizer>
        <acquisitionMethod>purchase</acquisitionMethod>
        <acquisitionNote>Acquisition authorizer role(s): Purchaser</acquisitionNote>
        <creditLine>Athenaeum purchase, 1881</creditLine>
    </ns2:acquisitions_common>
</document>

I am unable to duplicate the behavior that occurred during the batch ingest last night.

Spidey sense points toward collectionspace-mapper returning a truncated result when unable to get term lookup results from the CollectionSpace instance's API (due to scheduled restart or something?).

Need to test this and just have the record mapping fail if this is the case.

kspurgin commented 3 years ago

This is the datarow hash passed to collectionspace-mapper: 

{"creditline"=>"Athenaeum purchase, 1881",
  "accessiondategroup"=>"1881",
  "acquisitionmethod"=>"purchase",
  "acquisitionreferencenumber"=>"UU1881.1.1 (migrated accession)",
  "acquisitionauthorizer"=>"Charles Callahan Perkins (1823-1886)",
  "acquisitionnote"=>"Acquisition authorizer role(s): Purchaser"}

The order in which these fields are processed (i.e. the order in which they appear in the acquisitions RecordMapper) is:

This is a mystery because if failed API calls were the problem, that would have happened on accessiondategroup and acquisitionauthorizer.

But acquisitionmethod made it in the truncated record.

acquisitionnote and creditline are both non-repeatable simple fields at the top level of the namespace section, so why didn't they make it?