The Storage Service can run multiple tasks simultaneously. It might run multiple tasks more slowly because of CPU usage or disk and/or network bandwidth, but it does not block other tasks from running when one is already in process.
Current behaviour
Under some configurations, the Storage Service becomes unavailable to run additional tasks when one long task is already running. "Long" could be anywhere from a few minutes to a few hours.
By "tasks" I mean:
Send transfer to Archivematica
Store AIP (send to storage)
Fixity check on stored AIP
Download AIP
The problem can appear as one or more of the following, depending on configuration settings:
The Storage Service simply hangs and doesn't respond until the currently running task completes. A browser tab may simply hang and then load as soon as the task finishes, and API requests may do the same thing. I ran a fixity check on a 600 GB AIP and tried to access the Storage Service in a browser. After 5+ hours, the fixity check completed and the tab loaded.
The Archivematica dashboard may report that the Storage Service is not connected
The transfer dropdown on the Archivematica dashboard doesn't show any transfer sources
The Storage Service may return an Internal Server Error or a 404 if it does not simply hang
Attempts to store AIPs will fail
Steps to reproduce
This problem has existed in the Computer History Museum's installations of 1.8 and 1.9.1. It's proven very hard for me to reproduce when setting up local testing Archivematica installations. The specific way the Storage Service fails - either hanging or returning errors - seems to depend on the variables for Archivematica's configuration, and possibly on underlying server specifications (more or fewer CPUs/threads, HDD vs. SSD).
What seems to be consistent is that the Storage Service can fail unpredictably when asked to do multiple tasks concurrently. Ultimately, this issue may be the same as https://github.com/archivematica/Issues/issues/730 .
Your environment (version of Archivematica, OS version, etc)
Archivematica 1.8, 1.9, 1.9.1 on Ubuntu 18.04
This problem was not present on versions of Archivematica 1.3, 1.4, 1.5 or 1.6 running on Ubuntu 12.04 or Ubuntu 14.04.
For Artefactual use:
Please make sure these steps are taken before moving this issue from Review to Verified in Waffle:
All PRs related to this issue are properly linked 👍
All PRs related to this issue have been merged 👍
Test plan for this issue has been implemented and passed 👍
Documentation regarding this issue has been written and it has been added to the release notes, if needed 👍
sromkey
commented
5 years ago
Expected behaviour
The Storage Service can run multiple tasks simultaneously. It might run multiple tasks more slowly because of CPU usage or disk and/or network bandwidth, but it does not block other tasks from running when one is already in process.
Current behaviour Under some configurations, the Storage Service becomes unavailable to run additional tasks when one long task is already running. "Long" could be anywhere from a few minutes to a few hours.
By "tasks" I mean:
The problem can appear as one or more of the following, depending on configuration settings:
Steps to reproduce
This problem has existed in the Computer History Museum's installations of 1.8 and 1.9.1. It's proven very hard for me to reproduce when setting up local testing Archivematica installations. The specific way the Storage Service fails - either hanging or returning errors - seems to depend on the variables for Archivematica's configuration, and possibly on underlying server specifications (more or fewer CPUs/threads, HDD vs. SSD).
What seems to be consistent is that the Storage Service can fail unpredictably when asked to do multiple tasks concurrently. Ultimately, this issue may be the same as https://github.com/archivematica/Issues/issues/730 .
Your environment (version of Archivematica, OS version, etc)
Archivematica 1.8, 1.9, 1.9.1 on Ubuntu 18.04
This problem was not present on versions of Archivematica 1.3, 1.4, 1.5 or 1.6 running on Ubuntu 12.04 or Ubuntu 14.04.
For Artefactual use: Please make sure these steps are taken before moving this issue from Review to Verified in Waffle: