sul-dlss / libsys-airflow

Airflow DAGS for migrating and managing ILS data into FOLIO along with other LibSys workflows
Apache License 2.0
5 stars 0 forks source link

852$x mapping vs. migration tools #727

Closed ahafele closed 1 year ago

ahafele commented 1 year ago

The 852$x is mapped to a Holdings staff only note - see https://folio.stanford.edu/inventory/view/c38590d6-6491-5825-a323-a5e75484e59d/3fb73d0[…]d83-9a97-765459ba0ab4?qindex=hrid&query=a5499674&sort=title. For this record I modified the MARC, saved, and therefore sent back through mapping.

But on migration the 852$x is going to a holdings statement Public note - https://folio.stanford.edu/inventory/view/c6740076-a9fb-52d0-828a-df02261862c8/6613a9ea-bc51-519a-9910-17f2c3f06a77?query=a5487807&sort=title (edited)

jermnelson commented 1 year ago

This problem occurs because of how the folio_migration_tools parses the Holding Mapping rule for the 852 entity field. Our current Holdings Mapping Rule for the 852 looks like this:

[{'entity': [{'description': 'Public note',
              'rules': [{'conditions': [{'type': 'remove_ending_punc, trim'}]}],
              'subfield': ['z'],
              'target': 'holdingsStatements.note'},
             {'applyRulesOnConcatenatedData': True,
              'description': 'Holdings note type id',
              'rules': [{'conditions': [{'parameter': {'name': 'Note'},
                                         'type': 'set_holdings_note_type_id'}]}],
              'subfield': ['x'],
              'target': 'notes.holdingsNoteTypeId'},
             {'applyRulesOnConcatenatedData': True,
              'description': 'Note',
              'rules': [{'conditions': [{'type': 'trim'}]}],
              'subFieldDelimiter': [{'subfields': ['x'], 'value': ' ; '}],
              'subfield': ['x'],
              'target': 'notes.note'},
             {'applyRulesOnConcatenatedData': True,
              'description': 'If true, determines that the note should not be '
                             'visible for others than staff',
              'rules': [{'conditions': [], 'value': 'true'}],
              'subfield': ['x'],
              'target': 'notes.staffOnly'},
             {'description': 'Call number type, defined by 1st indicator',
              'rules': [{'conditions': [{'type': 'set_call_number_type_id'}]}],
              'subfield': [],
              'target': 'callNumberTypeId'},
             {'applyRulesOnConcatenatedData': True,
              'description': 'The permanent shelving location',
              'rules': [{'conditions': [{'type': 'set_permanent_location_id'}]}],
              'subfield': ['b', 'c'],
              'target': 'permanentLocationId'},
             {'applyRulesOnConcatenatedData': True,
              'description': 'Call Number identifier',
              'subFieldDelimiter': [{'subfields': ['h', 'i'], 'value': ' '}],
              'subfield': ['h', 'i'],
              'target': 'callNumber'},
             {'description': 'Prefix of the call number',
              'subfield': ['k'],
              'target': 'callNumberPrefix'},
             {'description': 'Shelving form of title',
              'subfield': ['l'],
              'target': 'shelvingTitle'},
             {'description': 'Suffix of the call number',
              'subfield': ['m'],
              'target': 'callNumberSuffix'},
             {'description': 'Range of numbers for copies that have the same '
                             'location',
              'subfield': ['t'],
              'target': 'copyNumber'}]}]

For this rule, the folio_migration_tools takes the first entity row to determine what Holdings Property to apply in this case the holdingsStatement. What we want instead is to use the notes property. Changing the order of these rules does create the proper note property for the Holding but may have other implications for different 852 fields in other MHLD MARC records.

ahafele commented 1 year ago

Typing thoughts here ahead of our meeting - If we reorder, I wonder then what happens to the 852$z for example. How is it handling the call number portion correctly then?

ahafele commented 1 year ago

Ryan reports at least 23545 852$x for Law

ahafele commented 1 year ago

This ckey has both 852x and 852z 1823064 This has just $z 13748517

jermnelson commented 1 year ago

For ckey 1823064 the original 854 field: =852 \\$aCSt-Law$bLAW$cFOLIO-BAS$t1$xli:rtw/li:cn$zLibrary no longer subscribes to this title or its supplements.

Changing the order of the rule results in the Holding record with a single note in the notes property with the subfield z value but is missing the subfield x value:

{'metadata': {'createdDate': '2023-08-04T21:09:31.385',
  'createdByUserId': '2c4429e2-3bc4-4117-9539-51bcf534f706',
  'updatedDate': '2023-08-04T21:09:31.385',
  'updatedByUserId': '2c4429e2-3bc4-4117-9539-51bcf534f706'},
 'id': '3b48f23f-dcb4-5abe-8a49-ead89bbb7876',
 'administrativeNotes': ['Identifier(s) from previous system: a1823064'],
 'formerIds': ['a1823064', 'Bib id: a1823064'],
 'instanceId': '42321776-d4f7-5a0b-b55a-834893cd9b72',
 'hrid': 'ho00000624482',
 'notes': [{'holdingsNoteTypeId': 'b160f13a-ddba-4053-b9c4-60ec5ea45d56',
   'note': 'Library no longer subscribes to this title or its supplements',
   'staffOnly': True}],
 'holdingsStatements': [{'statement': '1989,1992,1994'},
  {'statement': '1 v. <binder>'},
  {'statement': '"Bell\'s bugging book" 1 v'},
  {'statement': '"Bell\'s motion manual" 1 v',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'},
  {'statement': '"Bell\'s points & authorities" 1 v',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'}],
 'holdingsStatementsForSupplements': [{'statement': '"Bell\'s points & authorities suppl." 1 v',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'}],
 'callNumberTypeId': '95467209-6d7b-468b-94df-0f5d7ad2747d',
 'copyNumber': '1',
 'permanentLocationId': '982d671f-1f95-49ce-8878-05787c89cfde',
 'callNumber': 'MARC Holdings',
 'holdingsTypeId': 'e6da6c98-6dd0-41bc-8b4b-cfd4bbd9c3ae',
 'discoverySuppress': False,
 'sourceId': '036ee84a-6afd-4c3c-9ad3-4a12ab875f59'}

Here is the Holdings record for the last migration, note that the subfield z is not present at all in the record but the subfield x is present in the holdingsStatements.

{'metadata': {'createdDate': '2023-08-01T06:40:37.250',
  'createdByUserId': '2c4429e2-3bc4-4117-9539-51bcf534f706',
  'updatedDate': '2023-08-01T06:40:37.250',
  'updatedByUserId': '2c4429e2-3bc4-4117-9539-51bcf534f706'},
 'id': '4321123e-9dd7-5206-9ed8-41201cfc1ae1',
 'administrativeNotes': [],
 'instanceId': '42321776-d4f7-5a0b-b55a-834893cd9b72',
 'hrid': 'ah1823064_2',
 'holdingsStatements': [{'note': 'li:rtw/li:cn',
   'holdingsNoteTypeId': 'b160f13a-ddba-4053-b9c4-60ec5ea45d56',
   'staffOnly': True},
  {'statement': '1989,1992,1994'},
  {'statement': '1 v. <binder>'},
  {'statement': '"Bell\'s bugging book" 1 v.'},
  {'statement': '"Bell\'s motion manual" 1 v.',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'},
  {'statement': '"Bell\'s points & authorities" 1 v.',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'}],
 'holdingsStatementsForSupplements': [{'statement': '"Bell\'s points & authorities suppl." 1 v.',
   'note': 'Library has latest ed. only',
   'staffNote': 'Discard when replaced'}],
 'callNumberTypeId': '95467209-6d7b-468b-94df-0f5d7ad2747d',
 'copyNumber': '1',
 'permanentLocationId': '982d671f-1f95-49ce-8878-05787c89cfde',
 'callNumber': 'MARC Holdings',
 'holdingsTypeId': 'e6da6c98-6dd0-41bc-8b4b-cfd4bbd9c3ae',
 'discoverySuppress': False,
 'sourceId': '036ee84a-6afd-4c3c-9ad3-4a12ab875f59',
 '_version': 1}
ahafele commented 1 year ago

@jermnelson if I'm reading the above correctly, it seems like the reordering is not a solution. Should we discuss next steps post-standup today?

ahafele commented 1 year ago

There is a related issue of an added blank entry in the holdings details - https://folio.stanford.edu/inventory/view/2068d20c-5abb-583b-9a49-14d1051cf344/629e4e78-f5a9-5802-8a6d-cb0e5a9a16ab?qindex=hrid&query=ah6011031_3&segment=holdings&sort=title When I open in quickmarc and resave, it disappears. Not sure we care. It isn't causing a problem for SW. Might cause an MHLD export issue?