azavea / pfb-network-connectivity

PFB Bicycle Network Connectivity
Other
40 stars 11 forks source link

Increase analysis RAM allocation and fix provisioning #945

Closed KlaasH closed 1 year ago

KlaasH commented 1 year ago

Overview

The functional change here is increasing the memory allocation to the analysis Batch task to use the full RAM available on the i3.2xlarge instance we're using in the Batch compute environment. I had previously concluded that the jobs were only using about half of the 32GB we had allocated, but the biggest jobs have been crashing recently and it's possible memory is to blame. And there's no reason not to use all the memory on the instance.

I'm opening this as a draft PR because I already deployed the job definition update to production, so what's left is the changes I made in the course of spinning up my dev instance again and doing a limited deploy (I reused the containers, so only the services that have the job definition ID embedded in their parameters needed to change).

Here's what else is going on here:

Resolves #940 (hopefully)

Testing Instructions

When the time comes, this should be tested by:

Checklist

KlaasH commented 1 year ago

Closing this, since these changes got folded into PR #947.