Open mgeerdsen opened 2 years ago
The following jobs use the WORKING_STORAGE_FAST exclusively if available:
only use WORKING_STORAGE:
All these use exactly one storage area and make no decision based on size (as bagit does), so we only need one kind of storage. As WORKING_STORAGE_FAST is set to ephemeral storage, this means that nearly all jobs already use that and nearly none use the EFS.
If no jobs have had any trouble with the ephemeral storage, we should check the remaining job types and could get rid of one storage type (probably EFS).
After switching bagit creation to EC2/EBS we should check the storage usage of the fargate based worker nodes:
Also we should check if we can simplify the workernode scripts if we don't need to decide which storage area to use anymore.