[US] Books: migrate items

parent of #16 parent of #17 [CLOSED] parent of #18

Tasks

Prepare the migration of items (~300K items).
Identify the ones that need to be migrated as records.

Analysis

Items migration consists of 2 main objects:

Libraries
Items

Libraries

id (barcode), primary key, unique, not null
type, possible values:
- external: 43 libraries (ex. RERO, Nebis, Springer, ...), 0 items referencing them
- internal: 24 libraries (ex. CERN LHC Library, CERN EP Library, CERN ARC Library, CERN Courier, ...)
- main: 2 libraries, CERN Central Library and Other but 0 items referencing this last one
- hidden: 1 library, CERN Central Storage, 0 items referencing it
name, address, email, phone, notes

Comparing this legacy data model with the new data model, libraries will corresponds to different rooms or buildings, belonging to the unique Location. It means that Items will have a reference to a room and not to a location.

Migration

The proposed schemas for libraries are the followings:

1 single location:

{
"location": {
    "locid": "1", # id format to be defined
    "name": "CERN Central Library"
  }
}

multiple internal locations:

{
"internal_locations": [
    {
        "phone": "", 
        "ilocid": 0, # id format to be defined
        "name": "Legal Service Library", 
        "legacy_id": 3, 
        "address": "", 
        "notes": "", 
        "locid": "1", 
        "email": ""
    }, 
    {
        "phone": "", 
        "ilocid": 1, 
        "name": "CERN Central Library", 
        "legacy_id": 6, 
        "address": "CH-1211 Geneva 23", 
        "notes": "", 
        "locid": "1", 
        "email": "library.desk@cern.ch"
    }, 
    ...
}

The location -> internal location structure is needed because Invenio ILS, in case of multiple locations, will change the loan workflow to "transit" books between different locations.

Items

barcode, primary key, unique, not null
id_bibrec, reference to the document
id_crcLIBRARY, reference to the library
collection, possible values Monograph, Reference, Archives, Library, Conference, LSL Depot, Oversize, Official, Pamphlet, CDROM, Standards, Video & Trainings, Periodical. To be deleted #3.
location, a string for the UDC classification (ex. R 614(02) PAN blue, Blacksburg 1981, 530.145.29 MAN, 539.125 WOR, 621.3.02 NAI, ...)
description, Volumes/Series identifier, see below
loan_period, 4 weeks, 1 week, Reference
status: see below
number_of_requests: int, number of requests while item on loan
expected_arrival_date: related to acquisition
creation_date, modification_date

One item does not have the barcode, to be fixed.

Volumes/Series

Currently, volume or series definition is in the description field. A few numbers:

Description with v. or vol.: ~39K
Description without v. or vol.: ~5K (ex. "volume", "missing", "part. 1", "1978", "Part 2", "v 1")
Description is NOT NULL or NOT "-": ~58K
Description is NULL or "-": ~29K

Status

missing: ~12K -> MISSING
on shelf ~280K -> LOANABLE (maybe better name)
in binding: ~120 -> IN_BINDING
on loan: ~2.7K -> not needed, handled by Loan object
on order: ~160 -> related to acquisition
out of print: ~40 -> to keep?
not published: ~84 -> to keep?
cancelled: ~10 -> acquisition/ILL?
not arrived: ~260 -> acquisition/ILL?
untraceable: ~700 -> what is the difference with missing?
under review: ~1 -> to keep?
scanning: not existing for now, do you need?

Migration

The proposed schema for item is the following:

{
    "itemid": "1", # id format to be defined
    "docid": "1", # id format to be defined
    "ilocid": "1",  # id format to be defined
    "legacy_bibrec_id": "111111",
    "legacy_library_id": "3",
    "barcode": "83-0384-4",
    "classification": "530.24 SEI",
    "description": "",
    "status": "LOANABLE",
    "circulation_restriction": "Reference",
    "medium": "",
    "created": "date",
    "updated": "date"
}

Questions

Library

What is type external of libraries used for? Looks like ILL.
What is type hidden of libraries used for?
What is type = main, name =Other (0 items)?

Items

What is collection for?
Collection Periodical, there are 47K items, but they are not displayed in the interface to the user. For example: https://cds.cern.ch/record/229779. Take into account that we wanted to delete collections, see #3.
What is location for? UDC classification?
Can we have loan_period 4 weeks as default and then restrict for Reference for some items? Is 1 week really needed? (There are 19K items with 1 weeek)
Review all status.
3 Medium field wanted: how this affect loans? Possible values: --, online, paper, CD-ROM, DVD, VHS. Currently it does not exist, are you going to update 300K items?
number_of_requests: how do you use this today?
expected_arrival_date: related to acquisition ?

TO DO

Fix item that does not have barcode: attached BibRec https://cds.cern.ch/record/810479

CERNDocumentServer / cds-migrator-kit