search

omeka-s-modules / Osii

Import items from other Omeka S installations
GNU General Public License v3.0
2 stars 0 forks source link

PDOException: SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '4077-2993' for key 'item_item_set.PRIMARY' in /opt/omeka-s/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:112 #8

Closed C-Norton closed 1 year ago

C-Norton commented 1 year ago

On ver 1.1.0 of OSII, an import caused SQL integrity constraint violations when importing data that already partially existed within the system. Running Omeka S 3.2.3

We have a snapshot of the system after this took place. We recognize we are on an older version of both the plugin and the platform, but are not right now ready to upgrade. What details do you need to investigate, or must this be replicable on newer builds?

jimsafley commented 1 year ago

Do you have the backtrace for the error? It may help to troubleshoot the issue.

No, it does not have to be replicable on newer versions. What exact steps did you take before seeing the error? You mention that the data "already partially existed within the system." Was there an incomplete import? Any information you can pass on will be helpful.

C-Norton commented 1 year ago

Stack trace: https://github.com/C-Norton/osii-error/blob/main/Stacktrace

Initially, in the previous job, I had imported this same data, however, I had forgotten to set up a literal->literal field map, and got an error. So I deleted all the associated items and media, but did NOT delete the item set that had been created, leaving an [unnamed] item set in place. Previously, most of this data had actually been in place, and had recently been deleted (we're evaluating usage of Omeka S for deployment, so I'm doing a bunch of testing, so lots of import, evaluate, delete, import activity). About 2 hours after deleting the data, I reused the existing snapshot, made the import settings changes, and imported. This appeared to work, until about 30 minutes later when the omeka server ran out of memory and swap, resulting in a reboot

jimsafley commented 1 year ago

That's quite the backtrace, but unfortunately it doesn't help.

Given the nature of the error, it may have occurred during an item update operation due to some mis-mapping in the ItemItemSets resource mapper, but the API is typically more graceful in handling integrity constraint violations, so it's puzzling. Perhaps a previous import attempt put the data in some unstable state. It's very difficult pinpoint the problem without reproducing it, and I can't reproduce.

I'm concerned why it took two hours to delete the data and why the server ran out of memory after 30 minutes. That's not normal. Omeka is typically good at garbage collection.

In any case, I recommend deleting the import (and all resources created by the incomplete import), adding a new one, and trying again.

C-Norton commented 1 year ago

Jim,

For the resource management issue, is this something that the Omeka team would like to investigate further? If so, would providing the database be of use to you?

jimsafley commented 1 year ago

Your sizable backtrace exposed a bug that we're working on right now. It's probably not related to the "Integrity constraint violation" exception, but it's likely a contributor to the process running slowly and reaching the memory limit.