search

mnylc / islandora_multi_importer

This is a flexible, twig based, all cmodel, tabular data to islandora Object importer with optional ZeroMQ processing
GNU General Public License v3.0
16 stars 15 forks source link

Error state after update #53

Open Natkeeran opened 6 years ago

Natkeeran commented 6 years ago

I am using googleAPI-Update-WIP branch with 7c7aae7fa9043eddc013e2b51fe87befc37569c0 commit.

Using the update feature to update MODS. Running "update existing projects" seems to apply the updates as expected. However, it leaves the batch entries in Error state:

ID | State | Message | Parent | Set ID |  
-- | -- | -- | -- | -- | --
pid:41 | Error | Exception occured: Failed to ingest pid:41. | pid:collection | 6

The following message is in the Drupal log:

Failed to ingest object: pid:41
code: 500
message:

Documenting the issue here

DiegoPino commented 6 years ago

HI Nat, did you update batch module with my version? I have a pending pull to change the unique index for that db table, but completely overwhelmed by work sadly until monday.

let me know. Any other messages in the apache error.log?

Natkeeran commented 6 years ago

Hi @DiegoPino

No, I missed that step. Can you please point me to the updated batch module. Np, not urgent. I think the update still processes correctly.

No other error messages in the apache error.log.

DiegoPino commented 6 years ago

Hey @Natkeeran, so my currently in use branch is https://github.com/mnylc/islandora_batch/tree/7.x-ISLANDORA-2046

Right now its mostly in sync (means the good things of 7.x are there) and is backwards compatible, won't break your site

But: I'm right now fixing the whole islandora batch mechanic. There is a big issue on how islandora manages the ingest. Each individual object (in the set) goes to a general batch, but then the derivatives are a sub-batch of each ingest, which means if derivative generation fails for any reason you don't have control over that at all. And moreover, your failed, unfinished batch derivatives get stuck in a queue forever.

Also: I need to change this https://github.com/mnylc/islandora_batch/blob/7.x-ISLANDORA-2046/islandora_batch.install#L48

I feel the primary key needs to be the ID (PID) + the set id, means a combined one. That allows much more freedom and fewer warnings: because you can actually have on batch set that ingested an object still around + one batch set that updates objects in a different set, that points to the same PID.

Finally, this changes, as 7.x now also, allow for purging and reingesting.

let me know if this helps and anyway I can improve this module for you

best

McFateM commented 5 years ago

I need to study this issue thread a bit more, but wanted to freshen this discussion a bit and remind myself that I'm also seeing this "faux error" scenario when updating just the metadata in a set of objects.

More detail can be found in https://trello.com/c/j0kIkdIX.

DiegoPino commented 5 years ago

@McFateM sadly, you will have to use our Batch branch https://github.com/mnylc/islandora_batch/tree/7.x-ISLANDORA-2046 to make updating work. Its stable and won't touch anything else so you can still keep batch ingesting ZIP files using the many modules that provide that (but why would you like to do that??? jajajaj). If you can help me with getting that pull into Islandora then you will have that change on Vanilla Islandora, but i don't have hight hopes nor too much time to follow the path of being ZEN anymore.