search

library-data-platform / ldp

Provides an analytics capability for FOLIO libraries
Apache License 2.0
15 stars 9 forks source link

Error "value too long for type character varying(114)" during update #225

Open bbarnetteebsco opened 1 year ago

bbarnetteebsco commented 1 year ago

During a run of the LDP process, I received the following ERROR:

error: ERROR:  value too long for type character varying(114)

CONTEXT:  COPY ldp_circulation_loan_history, line 245611, column loan__action_comment: "Could not locate item in stacks. We had stacks management & two circ staff look at different times. ..."

This error was discovered while trouble shooting an issue that caused incomplete data for srs_marctab, inventory_holdings, and srs_records. It occurred, during the LDP update process, between ldp: updating audit_circulation_logs and ldp: updating po_receiving_history. And is believed to have affected ldp: updating inventory_holdings. The following day's LDP process ran normally without any changes being made.

The following discrepancies were noted between the LDP process that produced this error and the following successful LDP process. 2023-01-02T17:44:55.208-06:00 to 2023-01-02T17:48:18.700-06:00 4 minutes for ldp: updating inventory_holdings for the ERROR-run of LDP
2023-01-03T17:44:42.217-06:00 to 2023-01-03T20:01:58.083-06:00 2 hours 15 minutes for ldp: updating inventory_holdings for the normal-run of LDP

2023-01-02T17:48:18.700-06:00 to 2023-01-02T17:51:23.082-06:00 3 minutes for ldp: updating srs_records for the ERROR-run of LDP 2023-01-03T20:01:58.083-06:00 to 2023-01-03T20:31:26.712-06:00 30 minutes for ldp: updating srs_records for the normal-run of LDP

The total run time was 6 hours compared to the normal run time of 12 hours for this LDP process.

ldp_version: "1.8.2" ldpmarc_version: "v1.6.0" analytics_version: "v1.4.1"

nassibnassar commented 1 year ago

If the error might be reproducible, could you run ldp update with these options added to the command line?

--savetemps --table circulation_loan_history

That will try to update only the circulation_loan_history table, and it should leave behind a JSON file located under the data directory, in tmp/update/. Then see if you can find the JSON record within that file that triggered the error, by searching for the data string provided by the error context message. The string should be in the JSON field $.loan.actionComment.

If you can extract that JSON record or field, preserving any special characters etc., it could offer a clue.