pulibrary / bibdata

Local API for retrieving bibliographic and other useful data from Alma (Ruby 3.2.0, Rails 7.1.3.4)
BSD 2-Clause "Simplified" License
16 stars 7 forks source link

Error indexing numismatics #1717

Closed hectorcorrea closed 1 year ago

hectorcorrea commented 3 years ago

It looks like we are trying to save to Solr some documents without id when indexing numismatics. Below is the error reported:

W, [2021-10-18T12:56:54.279037 #11191]  WARN -- : Failed to index individual record , error was: RSolr::Error::Http: RSolr::Error::Http - 400 Bad Request
Error: {
  "responseHeader":{
    "status":400,
    "QTime":0},
  "error":{
    "metadata":[
      "error-class","org.apache.solr.common.SolrException",
      "root-error-class","org.apache.solr.common.SolrException"],
    "msg":"Document is missing mandatory uniqueKey field: id",
    "code":400}}

URI: http://lib-solr8-prod.princeton.edu:8983/solr/catalog-alma-production-rebuild/update?wt=json
Request Headers: {"Content-Type"=>"application/json"}
Request Data: "[{\"error\":\"Coin: 3386 with id: 817bf53f-d317-4962-a6ec-ad309c59fc94 has no parent numismatic issue and cannot build an OL document.\"}]"

Backtrace: /opt/marc_liberation/shared/bundle/ruby/2.6.0/gems/rsolr-2.3.0/lib/rsolr/client.rb:206:in `rescue in execute'
/opt/marc_liberation/shared/bundle/ruby/2.6.0/gems/rsolr-2.3.0/lib/rsolr/client.rb:196:in `execute'
/opt/marc_liberation/shared/bundle/ruby/2.6.0/gems/rsolr-2.3.0/lib/rsolr/client.rb:191:in `send_and_receive'
(eval):2:in `post'
/opt/marc_liberation/shared/bundle/ruby/2.6.0/gems/rsolr-2.3.0/lib/rsolr/client.rb:94:in `update'
/opt/marc_liberation/shared/bundle/ruby/2.6.0/gems/rsolr-2.3.0/lib/rsolr/client.rb:113:in `add'
/opt/marc_liberation/releases/20211018160758/app/services/numismatics_indexer.rb:29:in `block in index_individually'
/opt/marc_liberation/releases/20211018160758/app/services/numismatics_indexer.rb:28:in `each'
/opt/marc_liberation/releases/20211018160758/app/services/numismatics_indexer.rb:28:in `index_individually'
/opt/marc_liberation/releases/20211018160758/app/services/numismatics_indexer.rb:20:in `rescue in block in full_index'
/opt/marc_liberation/releases/20211018160758/app/services/numismatics_indexer.rb:17:in `block in full_index'

Notice the "id" is missing from the error message:

Failed to index individual record [THE-ID-SHOULD-HAVE-BEEN-HERE], error was: RSolr::Error::Http: RSolr::Error::Http - 400 Bad Request

Perhaps we should log these issues but as warnings rather than errors. We should also not try to push them to Solr in the batch update since it causes the batch to fail, and although, the individual records are eventually saved individually it's something that we could avoid and get better logging in general.

christinach commented 1 year ago

we haven't seen this issue for a long time