search

mnylc / islandora_multi_importer

This is a flexible, twig based, all cmodel, tabular data to islandora Object importer with optional ZeroMQ processing
GNU General Public License v3.0
16 stars 15 forks source link

Unable to import due to derivative generation failure #102

Closed bseeger closed 5 years ago

bseeger commented 5 years ago

Hi, I'm using ISLE and IMI to import a data set - mostly just testing it out. I keep running into this error message:

Screen Shot 2019-03-29 at 11 09 48 AM

This is islandora 7x-1.11.

I'm probably just doing something wrong, just not sure what.
Thanks! Bethany

bseeger commented 5 years ago

I should add that I'm running into this after I've uploaded the zip file and ask it to start processing the set.

bseeger commented 5 years ago

I should add here, that I added another column to the spreadsheet to add a new DS. This was so that I could have a jpg and tiff as masters - I called the 2nd DS TIF. I wonder if that's messing things up somehow?

bseeger commented 5 years ago

I was finally able to get the ingest to work, but the fix is kind of odd. I had ignored the compoundCModel section of the CMODEL mapping page. I had left the TN field at "--Don't Create--". When I did an ingest with that set to "Build using islandora generated derivative", the ingest worked fine. So, there might be a bug here. I'll see if I can reproduce it next week.

DiegoPino commented 5 years ago

Hi Bethany. Could you share a few lines of your spreadsheet data and your ingest setup, imi generates a watchdog message with your settings and mappings you can copy and share for that last part.

Since IMI does not really trigger derivaties by itself but just builds the objects batch expects and lets islandora do that, i will need to watch at that particular derivative. We have an history in islandora of building functions that not totally respect defined signatures when dealing with hooks, mostly because hooks are defined loosly in an api file but only tested in their functionality when invoked via drupal. I will give this a good look tomorrow. It could be something i can deal with at IMI level as i did for the context module or something i can push for core islandora

El El vie, 29 de mar. de 2019 a las 16:58, B. Seeger < notifications@github.com> escribió:

I was finally able to get the ingest to work, but the fix is kind of odd. I had ignored the compoundCModel section of the CMODEL mapping page. I had left the TN field at "--Don't Create--". When I did an ingest with that set to "Build using islandora generated derivative", the ingest worked fine. So, there might be a bug here. I'll see if I can reproduce it next week.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mnylc/islandora_multi_importer/issues/102#issuecomment-478147777, or mute the thread https://github.com/notifications/unsubscribe-auth/AGn854SnOf1sIlgPbUWtM-dld-4aprwLks5vbn6OgaJpZM4cS4aN .

-- Diego Pino Navarro Digital Repositories Developer Metropolitan New York Library Council (METRO)

DiegoPino commented 5 years ago

@bseeger hi. So looking at the code, this is one of the parts where this is invoked https://github.com/Islandora/islandora_solution_pack_compound/blob/e494516650c50de2ffe6fb0d68a8977c7f888a21/includes/derivatives.inc#L28 $child is passed by trying to load the $child object and it has a try surrounding the whole code. I would guess this derivative is making some assumptions that children are already which is sad. The order of ingest is always parents first and then children.

But also, maybe you are running some old code because the offending lines are not matching the ones in the repo right now. Could you look at those? See the lines don't match

https://github.com/Islandora/islandora_solution_pack_compound/blob/d914dec43ee8dc8c07afc44c6b78f8325b4efba3/islandora_compound_object.module#L367-L416 Also, maybe its because the strictness of the typing there. Its asking for an AbstractObject in child and Parent, but setting Child as NULL, so if you actually pass NULL (which would be the result of trying to load a child and not being able because its not there) you would be breaking the function signature contract. To me it looks like bad coding. Also a try around all the code assumes we are controlling somehow the Exceptions, but not an exception on the actual function signature.Again, bad coding.

I could not reproduce the error (i have all derivatives and compound things enabled in my test repo).

I could eventually force a thumbnail creation in IMI doing something like https://github.com/Islandora/islandora_solution_pack_compound/blob/e494516650c50de2ffe6fb0d68a8977c7f888a21/modules/islandora_compound_object_zip_importer/islandora_compound_object_zip_importer.module#L99

But then it would be hacking it, because derivatives really don't need to be run by the preprocessor, which is what IMI does.

@McFateM @patdunlavey Any chance you have seen this before? @bseeger finally, can you share more data as i requested to find the exact culprit?

Thanks

Diego

bseeger commented 5 years ago

Hi Diego, Thanks for looking into this - I will respond to your comments/questions above, but it will take me a few days because of a busy week. I think you'll hear from me again on Thursday.

DiegoPino commented 5 years ago

Perfect, no hurry. Thanks

El El lun, 1 de abr. de 2019 a las 16:28, B. Seeger < notifications@github.com> escribió:

Hi Diego, Thanks for looking into this - I will respond to your comments/questions above, but it will take me a few days because of a busy week. I think you'll hear from me again on Thursday.

— You are receiving this because you were assigned.

Reply to this email directly, view it on GitHub https://github.com/mnylc/islandora_multi_importer/issues/102#issuecomment-478734909, or mute the thread https://github.com/notifications/unsubscribe-auth/AGn85ybNx-Km6MQU0x90_gN5HpqMahehks5vcmv-gaJpZM4cS4aN .

-- Diego Pino Navarro Digital Repositories Developer Metropolitan New York Library Council (METRO)

bseeger commented 5 years ago

Hi @DiegoPino , just a quick update - I just got pulled into something today and now will not get back to this as soon as I planned. it will take a few weeks now for me to circle back to this. Sorry about the delay!

DiegoPino commented 5 years ago

@bseeger is this something you ever solved? There were some important updates on islandora compound derivative generation that could have solved this for you. Would love to follow up or close if not reproducible. We can always come back to this an reopen. Thanks!

bseeger commented 5 years ago

Hi @DiegoPino - sorry it took so long to get back to this. I just tried to reproduce it and am not seeing the issue anymore. I'll close this for now and re-open if I see it again.