sul-dlss / searchworks_traject_indexer

indexing MARC, MODS, and more for SearchWorks
Other
6 stars 1 forks source link

Transferred items should trigger reindexing of the source instance #980

Open cbeer opened 1 year ago

cbeer commented 1 year ago

Jeanette transferred the SAL item from instance a1888067 to instance a2010714. a2010714 was reindexed (because an item on the record had a recent updated timestamp), but a1888067 wasn't.

How are we supposed to know to reindex a1888067?

The same issue may apply to deleted items and holdings:

jcoyne commented 1 year ago

Possibly needs to be handled with a workflow change, so that the person who initiates the transfer updates both records.

shelleydoljack commented 1 year ago

I looked into this a bit. There are tables in sul_mod_inventory_storage, audit_instance, audit_item, audit_holdings_record, that I thought might contain something but I searched audit_item with the item UUID and audit_holdings_record with holding UUID from a1888067 and didn't get anything back:

okapi=# select * from sul_mod_inventory_storage.audit_item where jsonb#>> '{record,id}' = '141956d5-5ef8-5736-bd56-5090585f6e3d';
 id | jsonb 
----+-------
(0 rows)

okapi=# select count(*) from sul_mod_inventory_storage.audit_holdings_record;
 count 
-------
   136
(1 row)

okapi=# select * from sul_mod_inventory_storage.audit_holdings_record where jsonb#>> '{record,id}' = '7ca7c2bb-3198-50ae-ac1a-0a45c4560f00';
 id | jsonb 
----+-------
(0 rows)

I then searched the mod-inventory-storage repo for audit_item and it looks like maybe the tables are related to keeping track of deletes. The API docs for inventory-move show holdings can be moved as well. I'm not sure what the use case for moving holdings to a different instance would be but it could probably happen. And this transferring thing makes me wonder about when stuff is transferred to SAL3, would Caiasoft update the old holdings or instance with a note too (if that's the workflow route we must go to solve this issue)?

shelleydoljack commented 1 year ago

There are some kafka topics from inventory that I wonder if they might contain the info:

cluster.sul.inventory.holdings-record
cluster.sul.inventory.instance
cluster.sul.inventory.item

I moved item STA-12730460 (0186255d-dafa-5ed3-9a77-0d52a735edf2) from holdings record ah14193722_1 (c35bdf8b-7fbe-5907-92a3-f7ff51b381d9) to holdings record ah5680298_1 (17fd507f-93f9-5d10-8310-485f85ca914d) using the action "Move holdings/items to another instance" and looked at the kafka topics. There wasn't anything relevant in the instance or holdings-record topics but there was in item:

{
  "type": "UPDATE",
  "tenant": "sul",
  "old": {
    "instanceId": "515d026b-b94a-57df-8359-9d75ef62fa3b",
    "id": "0186255d-dafa-5ed3-9a77-0d52a735edf2",
    "_version": 1,
    "hrid": "ai14193722_1_1",
    "holdingsRecordId": "c35bdf8b-7fbe-5907-92a3-f7ff51b381d9",
    "formerIds": [],
    "discoverySuppress": true,
    "barcode": "STA-12730460",
    "effectiveShelvingOrder": "PR 46068 O93 H36 41999 11",
    "effectiveCallNumberComponents": {
      "callNumber": "PR6068.O93H36 1999",
      "typeId": "28927d76-e097-4f63-8510-e56f2b7a3ad0"
    },
    "yearCaption": [],
    "copyNumber": "1",
    "numberOfPieces": "1",
    "administrativeNotes": [],
    "notes": [],
    "circulationNotes": [],
    "status": {
      "name": "Available",
      "date": "2023-05-07T15:07:48.868+00:00"
    },
    "materialTypeId": "1a54b431-2e4f-452d-9cae-9cee66c9a892",
    "permanentLoanTypeId": "2b94c631-fca9-4892-a730-03ee529ffe27",
    "permanentLocationId": "08601677-5d55-459d-a499-b06114c940c2",
    "effectiveLocationId": "08601677-5d55-459d-a499-b06114c940c2",
    "electronicAccess": [],
    "statisticalCodeIds": [],
    "metadata": {
      "createdDate": "2023-05-07T15:37:55.084+00:00",
      "createdByUserId": "3e2ed889-52f2-45ce-8a30-8767266f07d2",
      "updatedDate": "2023-05-07T15:37:55.084+00:00",
      "updatedByUserId": "3e2ed889-52f2-45ce-8a30-8767266f07d2"
    }
  },
  "new": {
    "instanceId": "5de65e8a-aae5-5378-876c-00304c2da225",
    "id": "0186255d-dafa-5ed3-9a77-0d52a735edf2",
    "_version": 2,
    "hrid": "ai14193722_1_1",
    "holdingsRecordId": "17fd507f-93f9-5d10-8310-485f85ca914d",
    "formerIds": [],
    "discoverySuppress": true,
    "barcode": "STA-12730460",
    "effectiveShelvingOrder": "VROOMAN COLLECTION R 1",
    "effectiveCallNumberComponents": {
      "callNumber": "VROOMAN COLLECTION R",
      "typeId": "28927d76-e097-4f63-8510-e56f2b7a3ad0"
    },
    "yearCaption": [],
    "copyNumber": "1",
    "numberOfPieces": "1",
    "administrativeNotes": [],
    "notes": [],
    "circulationNotes": [],
    "status": {
      "name": "Available",
      "date": "2023-05-07T15:07:48.868+00:00"
    },
    "materialTypeId": "1a54b431-2e4f-452d-9cae-9cee66c9a892",
    "permanentLoanTypeId": "2b94c631-fca9-4892-a730-03ee529ffe27",
    "permanentLocationId": "08601677-5d55-459d-a499-b06114c940c2",
    "effectiveLocationId": "08601677-5d55-459d-a499-b06114c940c2",
    "electronicAccess": [],
    "statisticalCodeIds": [],
    "tags": {
      "tagList": []
    },
    "metadata": {
      "createdDate": "2023-05-07T15:37:55.084+00:00",
      "createdByUserId": "3e2ed889-52f2-45ce-8a30-8767266f07d2",
      "updatedDate": "2023-07-27T18:32:56.614+00:00",
      "updatedByUserId": "3e2ed889-52f2-45ce-8a30-8767266f07d2"
    }
  }
}

Maybe consuming this kafka topic might catch stuff? I look at this stuff by logging into one of the kafka pods in the k8s cluster and do kafka-console-consumer.sh --bootstrap-server kafka-folio-test.folio-test.svc.cluster.local:9092 --from-beginning --topic cluster.sul.inventory.item (see https://github.com/sul-dlss/folio-k8s/wiki/Kafka-topics).

thatbudakguy commented 1 year ago

Discussed at standup today with @ahafele the possibility that this could be a workflow thing, where the staff member adds a holdings note (or similar) to the instance that is losing its item.