search

jetbridge / cdk-nextjs

Deploy a NextJS application using AWS CDK
https://constructs.dev/packages/cdk-nextjs-standalone
Apache License 2.0
260 stars 43 forks source link

fix: delete from s3 in chunks #197

Closed jadenv closed 6 months ago

jadenv commented 6 months ago

When pruning old S3 objects, we ran into an issue where more than 1000 keys are attempted to be deleted at once which results in an error.

From AWS docs "The request can contain a list of up to 1000 keys that you want to delete." https://awscli.amazonaws.com/v2/documentation/api/latest/reference/s3api/delete-objects.html

This PR chunks the delete by batches of 1000 at a time to avoid the error.

V01D-NULL commented 6 months ago

bump This is causing our team quite the headache, would love to have this fix out soon

bestickley commented 6 months ago

@V01D-NULL, I haven't had time to test out this change myself. Have you tested out the fix? If it works for both of you independently, then I'm comfortable merging.

V01D-NULL commented 6 months ago

@bestickley I was able to build and upload jadenv's fork of the cdk to npm just to test it in our enviornment The bug fix works, and everything else went smoothly. It's an all clear from me!