Need to set Item Status, Material Type, Loan Type in FOLIO item records

[ahafele]

Related to #sul-dlss/folio-project-stanford/149

Per Darsi - need to add itemcat# values to figure out how to map to FOLIO

Related to item status - https://docs.google.com/spreadsheets/d/1QOFKqenPqMSOtsEODUVc9ILZQCViW4dHMfJSANYKg8U/edit#gid=0

[dlrueda]

Material type mapping for equipment https://docs.google.com/spreadsheets/d/1IF2rbz_qI8Mrmp-BPe0Ow19CpPoRHPxz/edit#gid=2048484277

I think it’s column A (item type) map to column G (FOLIO material type). Not sure if column C (library) is part of it.

Material type mapping for most Symphony itypes https://docs.google.com/spreadsheets/d/1TPLFtZfrtodjvoNv5xRuxHqUVhafq8xYZg6ZlnaGpL4/edit#gid=1910831552

This looks like column E (item type) maps to column D (FOLIO material type)

Material type mapping for Hoover items https://docs.google.com/spreadsheets/d/1u9EofQ5aPXtU60FH3E3Lz_DyTq_Efg71XVJpd2U-qhU/edit#gid=0

This adds in record format. We’ll want to add this to selitem/selcallnum/(and now selcatalog) output for .tsv files during Symphony export.

[shelleydoljack]

Updated do_catdumps_anditems*.ksh scripts to output catalog format to item tsv file. Also updated acquisitions/on_order_bibs/on_order_items_tsv.rb script with harcoded "MARC" value for format (since we won't be using catalog format to determine folio material type for on-order items).

[shelleydoljack]

Need mapping for on-order holding code item types to FOLIO material types. There are some item types in the Symphony holding codes to FOLIO locations mapping that aren't in the Material types in FOLIO mapping. I added Sheet3 which shows where we are missing mappings for item types (see column C where #N/A appears).

[shelleydoljack]

Need mapping for on-order holding code item types to FOLIO material types. There are some item types in the Symphony holding codes to FOLIO locations mapping that aren't in the Material types in FOLIO mapping. I added Sheet3 which shows where we are missing mappings for item types (see column C where #N/A appears).

I remember now why this matters: for creating the item tsv file for on-order bib records, we are using the mapping of Symphony holding code > item type that is in the spreadsheet Symphony holding codes to FOLIO locations mapping. So if there are item types in the item tsv for on-order bib records that are not in the Material types in FOLIO mapping, they will get "book" because that's currently our catch-all value.

[shelleydoljack]

Also, you'll want to look at the "SUL order types" tab in the Acquisitions mapping spreadsheet in order to align the on-order bibs' item types with order type material type, probably? For instance, material type database is missing from folio but part of the order type mapping and also one of the item types missing a material type mapping.

[shelleydoljack]

The red highlighted rows (with comments) in the "mapping" tab of List of equipment spreadsheet needs to be addressed.

[dlrueda]

@Laszlo-J can you comment on red highlighted rows in spreadsheet Shelley links to above?

[dlrueda]

Also, you'll want to look at the "SUL order types" tab in the Acquisitions mapping spreadsheet in order to align the on-order bibs' item types with order type material type, probably? For instance, material type database is missing from folio but part of the order type mapping and also one of the item types missing a material type mapping.

I’m checking with Suzette/Alissa on if we need to add the material types from the Acq mapping spreadsheet to folio-test in order to map on-order bib’s item types to them. @Laszlo-J fyi that we might want to add material types even though the Circ subgroup didn’t feel a need for them.

[Laszlo-J]

@shelleydoljack I made material type fixes and added comments to the red rows in the List of equipment spreadsheet.

I also updated and added a clean sheet for the Materials types in FOLIO Column A is the new FOLIO material type; B is the associated Symphony/Voyager item type.

[shelleydoljack]

OK, here's the material type mapping so far: https://github.com/sul-dlss/folio_migration/blob/t18-material-types/mapping_files/material_types.tsv If it looks okay, we can merge the PR and test it out.

[dlrueda]

Lines 9 and 10 are dups (probably because DVDCD is on the “Materials types in FOLIO” sheet twice

Lines 28 and 29 are dups with different FOLIO material types assigned (multimedia and book). Probably need @Laszlo-J to decide which it should be.

NH-7DAY is going to be problematic (and I see it isn’t in the map at all yet). It’s used by many libraries, 13,763 items have it as itype. Maybe we can map it depending on library? Something like NH-7DAY * EDUCATION maps to currshortloan7 (but this is really a LOAN TYPE)

[dlrueda]

@Laszlo-J is the equipment part complete in the material type mapping Shelley shows above?

There is a tab for equipment on the Material types in FOLIO spreadsheet: https://docs.google.com/spreadsheets/d/1TPLFtZfrtodjvoNv5xRuxHqUVhafq8xYZg6ZlnaGpL4/edit#gid=5623588

So far Shelley has made the mapping from the List of Equipment spreadsheet, but there are different/more itypes on the Materials types in FOLIO equipment tab. Do those need to be added to the mapping?

[Laszlo-J]

@shelleydoljack I updated the mapping sheet and addressed your comments: https://docs.google.com/spreadsheets/d/1IF2rbz_qI8Mrmp-BPe0Ow19CpPoRHPxz/edit#gid=1373351258

Also cleaned up the equipment type spreadsheet created by Circ: https://docs.google.com/spreadsheets/d/1TPLFtZfrtodjvoNv5xRuxHqUVhafq8xYZg6ZlnaGpL4/edit#gid=5623588

[shelleydoljack]

@Laszlo-J @dlrueda is there a decision about NH-7DAY yet? If we need to use library to map it to something specific, then that will require some code to be added to airflow. Also, MEDSTKS is mapped to two different FOLIO material type values. If library and record format are needed to determine which material type to assign, then we need to add code to airflow for that.

[shelleydoljack]

Also, you'll want to look at the "SUL order types" tab in the Acquisitions mapping spreadsheet in order to align the on-order bibs' item types with order type material type, probably? For instance, material type database is missing from folio but part of the order type mapping and also one of the item types missing a material type mapping.

I’m checking with Suzette/Alissa on if we need to add the material types from the Acq mapping spreadsheet to folio-test in order to map on-order bib’s item types to them. @Laszlo-J fyi that we might want to add material types even though the Circ subgroup didn’t feel a need for them.

@dlrueda Any new mappings surfaced yet for the on-order bibs item types to FOLIO material types? As far as I can tell, we are missing mappings for when the on-order bib will have these item types: CLUSTER, DATABASE, EDI-REMOVE, INDEX, LIBUSEONLY, MEDICAL, NH-7DAY, NH-INHOUSE, NH-MEDSTKS, NH-RESERVS, NH-RSVIDEO, PAGE-1DAY, REF, STOR, SUL, TIMO-COLL, UNKNOWN.

[dlrueda]

I have updated columns G and H of https://docs.google.com/spreadsheets/d/1TPLFtZfrtodjvoNv5xRuxHqUVhafq8xYZg6ZlnaGpL4/edit#gid=1587820449 and it is ready to be used.

There are still 4 item types that don’t map cleanly to one FOLIO material type: LIBUSEONLY, NH-7DAY, NH-INHOUSE, NH-MEDSTKS. These 4 are not in the column G/H mapping (I left them out). Let them default to “book” for the on-order bib mapping, and then the location will determine how they circulate once received. Or if @Laszlo-J wants to consult with Catherine/Kelly and see if they have a better default, that’s fine.

[shelleydoljack]

@dlrueda many thanks. Here's the updated mapping (draft PR) that we will use. For the 4 item types that need a mapping, if there's a better default (e.g. material type = migrate error?, otherwise book is the catchall) then we can add that before we attempt to load a bunch to folio-test. @Laszlo-J , for Hoover and HV-ARCHIVE, we are using item type, catalog format, and library to map material type (see lines 88-143). If you wanted to do something similar for LIBUSEONLY, NH-7DAY, NH-INHOUSE, NH-MEDSTKS then we'd have to add some code to libsys-airflow.

[Laszlo-J]

@shelleydoljack I looked at a sampling of these item types, and you should do the same thing with LIBUSEONLY, NH-7DAY, NH-INHOUSE, NH-MEDSTKS. There's too many records with too many different formats to assign a single one to each...

NH-MEDSTKS in particular makes me sad. :( (We were supposed to clean up this old item type when Media became it's own library, but it's still used all over the place by other libraries, and not just for audio-visual!) You could maybe migrate this as audiovisual, but I found examples of books and serials with this item type, so safer to use your schema to catch the weird ones...

[Laszlo-J]

@shelleydoljack I've updated the mapping for the spreadsheets here. The only real gotcha is LIBUSEONLY and RUMSEYMAP. There are dozens (hundreds?) of MARC cat format that should be material type = map or archival. Don't know how/if we handle those. Assuming they should be changed manually after import into FOLIO.

[shelleydoljack]

@Laszlo-J and @dlrueda I created a new mapping for material types. I did the mapping work on this spreadsheet: material type mapping worksheet. Take a close look at the differences in this draft PR. Are we not using "software" and "database" material types anymore? Do we need to remove these material types from folio? See material-types.tsv from FolioOps migration files. Also, it looks like there are duplicate ITEM_TYPEs mapped to different folio_names which isn't going to work.

[shelleydoljack]

Current draft PR for material and loan type mappings: https://github.com/sul-dlss/folio_migration/pull/61

• [x] Fix the duplicate ITEM_TYPE values mapped to different material types in material type mapping spreadsheet
• [x] Resolve use of "software" and "database" material types (was in version 1 of mapping but not in version 2).
• [x] Is the loan type mapping complete?

[dlrueda]

Software and database : Acq might want to use them when they order, but the original idea was that circ could set the material type to whatever it liked since it was only used for circulation, and they like electronic resource. I think it’s fine to have it map like that.

I am nervous about deleting default reference data, because I have read in a couple places that for upgrades, it gets re-created anyway, or FOLO is sometimes looking for some of this stuff and it’s a problem if it isn’t there. If someone knows better, that’d be cool.

[dlrueda]

Loan type mapping ticket is complete, nothing more to add. I did add one more row to the loan type mapping worksheet you link to above.

[shelleydoljack]

Software and database : Acq might want to use them when they order, but the original idea was that circ could set the material type to whatever it liked since it was only used for circulation, and they like electronic resource. I think it’s fine to have it map like that.

👍

I am nervous about deleting default reference data, because I have read in a couple places that for upgrades, it gets re-created anyway, or FOLO is sometimes looking for some of this stuff and it’s a problem if it isn’t there. If someone knows better, that’d be cool.

software and database are not folio reference data (we add those material types).

[shelleydoljack]

Loan type mapping ticket is complete, nothing more to add. I did add one more row to the loan type mapping worksheet you link to above.

Ok, updated the PR.

[dlrueda]

software and database are not folio reference data (we add those material types).

Ah, so, so then Acq wants them at point of order. So we’ll keep them for that reason.

[shelleydoljack]

Done for material types and loan types. Item status mapping is on https://github.com/sul-dlss/folio_migration/pull/63.