Staging server shuts down when I try to add a .zip file

[orangewolf]

Designs

https://gitlab.com/notch8/louisville-hyku/-/issues/67/designs/design_1650566447498.png

https://gitlab.com/notch8/louisville-hyku/-/issues/67/designs/design_1650566466110.png

[orangewolf]

Related Merge Requests

i67- add decorator to override csv_entry #possible_collection_ids

[orangewolf]

Rachel February 2022

I want to test a larger import (multiple records, each with multiple files) to see whether it works/if it creates the errors we've been seeing on our own production server for large batches of single-image records. Since we didn't provide you a sample set of an entire collection, I had to attempt to use the zip method. I've tried twice, and each time the staging server becomes unavailable after I Create and Validate the Importer; when I get back in to the staging server the importer I'd attempted to create and validate isn't there at all.

[orangewolf]

Shana February 2022

@rachel.howard could you attach the file you are trying to import, to the description of this ticket?@rachel.howard could you attach the file you are trying to import, to the description of this ticket?

[orangewolf]

Rachel February 2022

The file I was trying to import yesterday (Simmons) is too big. Here's another one I tried this morning that didn't break the sandbox but hasn't yet produced any works.

[orangewolf]

Rachel February 2022

Now it says this one is too big too! (Although it is much, much smaller). It is the Burt the Cat Fanclub Newsletter collection (Louisville Underground Music Archive Collection importer).

I can break it down into a smaller subcollection but I do want to be able to test the type of scale we'll actually be working with and it seems that the Sandbox doesn't have the capacity to do that (and I wonder whether our Production server does).

[orangewolf]

Shana February 2022

@rachel.howard Understood. I checked staging's sidekiq and it seemed like it was stuck in a weird state where it scheduled all the jobs to run later, instead of at the time the importer was created.

I'm not sure how this happened but will keep an eye on it to find the source of the issue and resolve it.

For now, I reran this importer (which I assume is the one you were trying?) but if not, please also link the importers you're attempting as well. As much info we can get is helpful to troubleshoot the issues, like: videos, screenshots, links, sample files, etc.

(UPDATE - I see you referenced the name of it in your comment - I found the importer)

If possible, could you put the LUMA_Burt.zip test files on your server and link us to download them, like we did w the initial sample files (if they are too large to upload here)? I reran that importer and can see that it immediately moves all the jobs into scheduled to run later, so I'd like a copy of this file to reproduce the behavior locally and fix it.

[orangewolf]

Rachel February 2022

Try these links:

1. SIMMONS - https://webservices.library.louisville.edu/hyku_csv/zipfiles/simmons.zip
2. LUMA BURT - https://webservices.library.louisville.edu/hyku_csv/zipfiles/LUMA_Burt.zip

[orangewolf]

Shana February 2022

Thanks for sharing the files. I am able to reproduce this behavior locally and will investigate what's going on.

[orangewolf]

Shana March 2022

re run after fix: http://lv-hyku-staging.notch8.cloud/importers/22?locale=en

[orangewolf]

Rachel March 2022

Cool! But, there are new bugs...

[orangewolf]

Alisha March 2022

@rachel.howard I assume you created tickets for the new bugs? either way, are you still experiencing the staging server shut down when you import a zip file?@rachel.howard I assume you created tickets for the new bugs? either way, are you still experiencing the staging server shut down when you import a zip file?

[orangewolf]

Rachel March 2022

I haven't tried to upload a zip file lately (it's not my preferred method, but I was desperate!), and haven't tried to upload anything this week since the staging server was being worked on.

[orangewolf]

Alisha April 2022

tested this out today no problem.

• importer: https://lv-hyku-staging.notch8.cloud/importers/9?locale=en
• zip: warriors_3_children.zip

collection	parent