Support zero-downtime deployments

[ronaldtse]

Due to the issue in #29 , @strogonoff and I discussed about separating the website upload and website activation phases.

Assume that we have this S3 bucket layout:

|--- 20210101000000
       |---  .done
       |--- index.html
       |--- foo
             |--- bar.html
|--- 20210201000000
       |---  .done
       |--- index.html
       |--- foo
             |--- bar.html
|--- 20210301000000
       |---  .done
       |--- index.html
       |--- foo
             |--- bar.html

• Each directory (directory = different path, since S3 has no concept of directories) contains a complete website uploaded at a different date.
• We use a Lambda@Edge function to always read the index.html from the newest bucket (the directory is named according to the user's preference, e.g. timestamp of initial upload). The .done file is created when the bucket is fully uploaded.
• The Lambda@Edge function should be integrated with the function specified in #29 for index path rewrites.

Then the upload flow would be:

1. User uploads complete website to a new path, e.g. 20210401000000/.
2. Once upload is complete, the user creates 20210401000000/.done.
3. User runs Cloudfront invalidation on the site. This means that new requests will hit the Lambda@Edge function.
4. A request to / hits Lambda@Edge, and the function detects that the newest (sorted latest) directory is 20210401000000 and it also contains the .done path.
5. Lambda@Edge returns the S3 file 20210401000000/index.html.
6. We can have a Lambda function that cleans up previous versions and keep only the latest site with .done or the last two copies every hour or every day.

This would be a good way of doing zero-downtime deployments.

Due to the additional complexity of this solution, this might better be a separate terraform module that builds on the current one.

Thoughts @strogonoff @phuonghuynh ?

[ronaldtse]

Ping @skalee since this is relevant to you too.

[ronaldtse]

@phuonghuynh could you let us know how this is going? Thanks.

[phuonghuynh]

I think we need to update current lambda-redirect code to support this feature An other reference link is

[ronaldtse]

@phuonghuynh what's the lambda-redirect code?

[phuonghuynh]

@ronaldtse this repo, https://github.com/riboseinc/terraform-aws-lambda-redirect-http-index, we can update code to read S3 objects and perform redirect to objects in latest .done file

[ronaldtse]

Sure, let's do that.

[phuonghuynh]

@ronaldtse how about using SQL Select feature of S3? https://docs.aws.amazon.com/AmazonS3/latest/userguide/selecting-content-from-objects.html

This enable us to save S3 cost; but we need to define a JSON object (max 1MB) for querying data we need.

[ronaldtse]

@phuonghuynh do you mean that we can use a CSV file for index and use S3 Select to return the correct path?

[phuonghuynh]

yep, correct, but I think in JSON format would be better for human fast reading?

[phuonghuynh]

@ronaldtse the implement code now using CSV format as bellow

lastUpdatedTimestamp,rootDir
1615877411,dir1
1615875912,dir2

When upload progress done, this S3 Config file need to be appended. Required env variables needed for lambda to load it.

- `BUCKET_NAME` (optional): bucket name to set CSV Config File
- `BUCKET_CSV_CONFIG_KEY` (optional): csv config bucket object, sample as bellow

PR sent, https://github.com/riboseinc/terraform-aws-lambda-redirect-http-index/pull/2 Need someone able to test with staging test site

[ronaldtse]

yep, correct, but I think in JSON format would be better for human fast reading?

Agree.

[phuonghuynh]

@ronaldtse I switch to use CSV because it easier than JSON when appending new line