search

harvard-library / aspace-import-excel

Import resources, archival objects, etc. into ArchivesSpace using Excel spreadsheet files
Apache License 2.0
43 stars 10 forks source link

Compatibility with AS v2.7.0? #61

Closed noahgh221 closed 4 years ago

noahgh221 commented 4 years ago

Since upgrading ArchivesSpace to v2.7.0, I have noticed that when I trigger a spreadsheet import from a resource record, I receive a Bad Gateway error in the staff interface (after a considerable delay).

I also see the following "Calling a deprecated endpoint" error in my log file (logs/archivesspace.out):

image

The plugin was working as expected in v2.6.0, so I'm wondering if something has changed in v2.7.0 that makes the current plugin incompatible (and if it's an easy fix:)). Thanks for considering.

kschlottmann commented 4 years ago

FWIW, I just tested a small import via the plugin on our 2.7.0 dev instance (we are hosted by Lyrasis), and it imported correctly.

noahgh221 commented 4 years ago

Thanks @kschlottmann . Looking into this a bit more, I was able to get the import to work (there were some spreadsheet formatting issues..). Even so, I do still get the "CALLING A DEPRECATED ENDPOINT" warning in my logs, but it doesn't seem like a critical error.

pobocks commented 4 years ago

I'll take a look and see which endpoint - I can't devote work time to fixing it right now, but either Bobbi or I can probably figure a replacement for it before it gets actually removed.

On Tue, Nov 19, 2019 at 9:04 AM Noah Huffman notifications@github.com wrote:

Thanks @kschlottmann https://github.com/kschlottmann . Looking into this a bit more, I was able to get the import to work (there were some spreadsheet formatting issues..). Even so, I do still get the "CALLING A DEPRECATED ENDPOINT" warning in my logs, but it doesn't seem like a critical error.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/harvard-library/aspace-import-excel/issues/61?email_source=notifications&email_token=AAAIH4EUDDZE3KUPQTZUVPLQUPW6HA5CNFSM4JO2OCPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEOJMHA#issuecomment-555521564, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAIH4HQTLLHQ72CBWUU7LTQUPW6HANCNFSM4JO2OCPA .

noahgh221 commented 4 years ago

thanks @pobocks . To add another wrinkle, Lyrasis was reporting that ASpace crashed every time we tried to import the malformed spreadsheet. That's why I started looking into this. Seems weird that a malformed spreadsheet would crash ASpace entirely rather than just throw an error. In the meantime, I will try to be more vigilant in enforcing spreadsheet formatting standards (wish me luck).

pobocks commented 4 years ago

Oh, wow. That's concerning - though I guess "unhandled exception" might well do it.

On Tue, Nov 19, 2019 at 9:54 AM Noah Huffman notifications@github.com wrote:

thanks @pobocks https://github.com/pobocks . To add another wrinkle, Lyrasis was reporting that ASpace crashed every time we tried to import the malformed spreadsheet. That's why I started looking into this. Seems weird that a malformed spreadsheet would crash ASpace entirely rather than just throw an error. In the meantime, I will try to be more vigilant in enforcing spreadsheet formatting standards (wish me luck).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/harvard-library/aspace-import-excel/issues/61?email_source=notifications&email_token=AAAIH4DCWJBFS7K4UDKY5JLQUP43TA5CNFSM4JO2OCPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEOO3NY#issuecomment-555543991, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAAIH4F3WX3IDI5AZIG5KXLQUP43TANCNFSM4JO2OCPA .

bobbi-SMR commented 4 years ago

I suspect that the deprecated route is being invoked by the javascript that helps determine which leaf of the tree is used as the starting point for the ingest.

In terms of "crashing the system": I'd really like to see the related log file(s).

noahgh221 commented 4 years ago

@bobbi-SMR , I forwarded you a message containing a snippet of the log file along with some other info. Thanks!

bobbi-SMR commented 4 years ago

Hi, Noah,

I ran into a problem with the excel file you sent that I can't explain: When trying to ingest the file, the code didn't get past the "read in and parse the excel file" piece, resulting in the following message in the log file: F, [2019-11-20T10:46:42.871000 #12100] FATAL -- : RuntimeError ({"error":"undefined methodgetSQLState' for java.lang.OutOfMemoryError: Java heap space:Java::JavaLang::OutOfMemoryError\nDid you mean? getClass"} ` (This, btw, might have caused Aspace to crash).

I opened the file, wrote it out as a csv, read that csv back in, and tried again. There were no problems.

I don't have any expertise in evaluating .xlsx files for problems, but I'm wondering if whatever the problem is is the root cause. I did all this in 5.2, as I'm only now in the process of upgrading myself, but I doubt that the file as you sent it to me is going to get past the parsing step.

[thanks for your kind words wrt the plugin :-)]

noahgh221 commented 4 years ago

Interesting, I will never understand Excel's mysteries... Thanks for the additional info though. I was able to get this spreadsheet to load after reformatting it to meet the template specs. I'll close out this issue and chalk it up to...Excel being Excel.