search

guardian / grid

The Guardian’s image management system
https://www.theguardian.com/info/developer-blog/2015/aug/12/open-sourcing-grid-image-service
Apache License 2.0
1.44k stars 120 forks source link

[reaper] re-write lambda #4135

Closed twrichards closed 11 months ago

twrichards commented 1 year ago

Re-write the existing lambda, primarily to coordinate (and log) calls to the endpoints added in https://github.com/guardian/grid/pull/4134. It's designed to run on a schedule (currently every 15mins, regardless of stage).

Each invocation it attempts to...

...logging the above and storing a permanent record to dedicated (& stage specific) S3 bucket.

* batch size is different per stage (proportional to env's ingenstion rate) but is 1000 for PROD (since this is the maximum batch for the bulk delete APIs etc. used under the hood).

twrichards commented 11 months ago

Closing in favour of a consolidated approach entirely within thrall - see https://github.com/guardian/grid/pull/4145