moved to postgres functionality in cleanup executions

etcart commented 2 weeks ago

Summary: update cleanExecutions Lambda to correctly interact with the database and clean up executions. write tests that more thoroughly test the internal workings

Addresses CUMULUS-3385 Cumulus P3/V16+ is not correctly running cleanExecutions

Changes

cleanExecutions lambda rewritten to interface with Postgres
tests added that put executions in test DB and check direct expected functionality
added UPDATE_LIMIT env configuration to allow batch processing expecting large backlog

PR Checklist

[x] Update CHANGELOG
[x] Unit tests
[x] Ad-hoc testing - Deploy changes and test manually
[ ] Integration tests

Jkovarik commented 2 weeks ago

@etcart I haven't pulled this for a formal review - this looks to be handling the RDS portion of the fix, is the ES component of the solution intended as another PR?

etcart commented 2 weeks ago

I had misunderstood and thought that this was meant to be independent of that

etcart commented 2 weeks ago

ok, now with ES and Postgres both. I haven't solved the problem of "too many executions" since that's a separate ticket, but I can't if we'd prefer to do it here

nasa / cumulus

moved to postgres functionality in cleanup executions #3691

Changes

PR Checklist