Create batch script to find documents with expiring s3 presigned urls and update them in the database
Create new Eventbridge scheduler schedule that runs every 3 hours to refresh s3 presigned urls
Create new S3_PRESIGNED_URL_RENEWAL_THRESHOLD environment variable to control when an s3 presigned url should be refreshed. Defaults to 7 hours.
Update participant ECS task role's max session duration to 12 hours
Update participant ECS task log level in dev environment to info
Fix flaky participant e2e test
Context for reviewers
Testing instructions, background context, more in-depth details of the implementation, and anything else you'd like to call out or ask reviewers. Explain how the changes were verified.
Because we weren't able to generate s3 presigned urls in lowdefy when the staff member logs into the staff portal, we instead generate them when the participant uploads the document in the participant portal and save the urls to the database, which are then retrieved by lowdefy when the staff member logs in.
We previously thought that we could set the maximum expiration time for these presigned urls to be 7 days. However, it turns out that the 7-day maximum is only for IAM users, not IAM roles. We're using a ECS Task IAM role to generate these presigned urls. S3 presigned url expiration time is tied to IAM role maximum session duration. Our ECS Task IAM role had a maximum session duration of 1 hour (AWS default).
This was causing staff users to be unable to access uploaded documents in the staff portal because the s3 presigned urls expired way too fast.
This PR addresses this with the following:
It increases the maximum session duration for the participant ECS Task IAM role to 12 hours (AWS maximum for IAM roles)
It adds a script to query the participant database for any documents that have not been updated within the last 7 hours ago, generates a new s3 presigned url, saves the new url to the database
It runs this script using an Eventbridge Schedule as an ECS Task on a recurring basis every 3 hours
This is run every 3 hours looking back 7 hours for added redundancy. The script will ignore documents that have been updated within the last 7 hours.
Testing
Screenshots, GIF demos, code examples or output to help show the changes working as expected. ProTip: you can drag and drop or paste images into this textbox.
Test locally:
Set the S3_PRESIGNED_URL_RENEWAL_THRESHOLD env var to something really low (like 120 seconds)
Create a submission in the participant portal
Log into the staff portal
Verify that the presigned url works
Wait until the expiration time passes
Verify that the presigned url expired and no longer works
Run the script npm run refresh-s3-urls
Refresh the staff portal
Verify that the new presigned url works
Tested on dev:
I verified that when the participant ECS Task maximum session duration was 1 hour, presigned urls expired after 1 hour
I verified that when the participant ECS Task maximum session duration was 1 hour, running the refresh script, successfully refreshed presigned urls
I verified that after increasing the participant ECS Task maximum session duration to 12 hours, presigned urls were good for longer than 1 hour
Ticket
Changes
S3_PRESIGNED_URL_RENEWAL_THRESHOLD
environment variable to control when an s3 presigned url should be refreshed. Defaults to 7 hours.info
Context for reviewers
Because we weren't able to generate s3 presigned urls in lowdefy when the staff member logs into the staff portal, we instead generate them when the participant uploads the document in the participant portal and save the urls to the database, which are then retrieved by lowdefy when the staff member logs in.
We previously thought that we could set the maximum expiration time for these presigned urls to be 7 days. However, it turns out that the 7-day maximum is only for IAM users, not IAM roles. We're using a ECS Task IAM role to generate these presigned urls. S3 presigned url expiration time is tied to IAM role maximum session duration. Our ECS Task IAM role had a maximum session duration of 1 hour (AWS default).
This was causing staff users to be unable to access uploaded documents in the staff portal because the s3 presigned urls expired way too fast.
This PR addresses this with the following:
This is run every 3 hours looking back 7 hours for added redundancy. The script will ignore documents that have been updated within the last 7 hours.
Testing
Test locally:
S3_PRESIGNED_URL_RENEWAL_THRESHOLD
env var to something really low (like 120 seconds)npm run refresh-s3-urls
Tested on dev: