collectionspace / cspace-converter

Migrate data to CollectionSpace.
0 stars 7 forks source link

Bump versions upd cache #361

Closed mark-cooper closed 4 years ago

kspurgin commented 4 years ago

@mark-cooper -- When I tried to use this branch to load 100 collectionobject records (having name, place, and organization authorities and at least one vocabulary term), 7 of them failed to import.

The error I get in console is not super helpful, and is the kind of thing we were seeing with some of the XML-creation logic:


D, [2020-06-25T16:29:08.603155 #72343] DEBUG -- : Generating content for: CollectionSpace::Converter::Lhmc::LhmcCollectionObject -- {"objectnumber"=>"PSF 443.004", "objectproductionorganization"=>"Elliott's Peachtree Studio", "objectproductionorganizationrole"=>"maker", "briefdescription"=>"Philip Shulhafer.", "objectname"=>"Photograph", "numberofobjects"=>"1", "material"=>"Gelatin silver print", "objectproductionplacelocal"=>"Atlanta, Georgia", "objectproductiondategroup"=>"1943-1944", "comment"=>"Original.", "namedcollection"=>"Philip Shulhafer Family Papers", "dimensionsummary"=>"overall: 6 3/4 in x 4 7/8 in", "limitationtype"=>"lending", "limitationlevel"=>"restriction", "collection"=>"permanent-collection", "publishto"=>"All", "inventorystatus"=>"processed"}, {:id=>"collectionobjects", :identifier_field=>"objectNumber", :path=>"collectionobjects", :schema=>"collectionobjects", :identifier=>"PSF 443.004", :title=>"PSF 443.004"}
W, [2020-06-25T16:29:08.604242 #72343]  WARN -- : Multivalued fields used in measuredPart Group have uneven numbers of values
W, [2020-06-25T16:29:08.604402 #72343]  WARN -- : Multivalued fields used in measuredPart Group have uneven numbers of values
W, [2020-06-25T16:29:08.604574 #72343]  WARN -- : Multivalued fields used in dimension within measuredPart (#1) have uneven numbers of values
W, [2020-06-25T16:29:08.604722 #72343]  WARN -- : Multivalued fields used in dimension within measuredPart (#1) have uneven numbers of values
E, [2020-06-25T16:29:09.056407 #72343] ERROR -- : Error for import row: undefined method `[]' for nil:NilClass
E, [2020-06-25T16:29:09.056731 #72343] ERROR -- : Error for import row: undefined method `[]' for nil:NilClass

However, if I switch back to master (and rename my ~/.cspace-converter directory) and restart rails, all import with no failures.

Not sure how to track down exactly what the problem is. I will send you the file I am using to test, since it is real client data. (These records do not exist in Breman's CSpace test instance yet -- they have not been transferred, only imported)

mark-cooper commented 4 years ago

@kspurgin this is fixed here: https://github.com/collectionspace/collectionspace-refcache/blob/master/lib/collectionspace/refcache/search.rb#L12

kspurgin commented 4 years ago

Re-tested and all 100 imported and transferred as expected. Seems good to go to me...

mark-cooper commented 4 years ago

@kspurgin I appreciate you looking at it so quickly, thx.

It's a little bit buried right now but if you want to preload the cache from a file you can still do that:

  1. Remove old cache files.
  2. Run bundle exec rake cache:download to generate a new cache file.
  3. On startup if the cache file is found it will be preloaded. It's not kept in sync in any way, so step 2. has to be rerun manually whenever needed.

When importing data for an empty system it would make sense to turn the cache search feature off, so we'll probably want runtime config around that behavior soon.