search

mdn / stumptown-deployer

🚢 those static Stumptown sites
Mozilla Public License 2.0
4 stars 6 forks source link

Copy from a "master" S3 bucket first #4

Open peterbe opened 5 years ago

peterbe commented 5 years ago

Realistically, we'll have a "master" S3 bucket that gets used for things like hosting the final production site. Most possibly, someone makes a small edit that only affects a minority of all the files. For example, they edited the abbr/docs.md in stumptown-content and now the only files that are truely different are docs/Web/HTML/Element/abbr/index.html and docs/Web/HTML/Element/abbr/index.json. It would be inefficient to have to upload all thousands of files when it turns out that it's identical to the master bucket except these two files.

Mind you, it might be that every single file is different too. For example, if the JS bundle changes from static/js/main.780e3893.chunk.js to static/js/main.89af470b.chunk.js then every single index.html generated will be different.

peterbe commented 5 years ago

As a possible solution, we can use this: https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.copy

If you can know which S3 bucket is the master you can first, before doing the uploads and after having created the bucket, run that copy command.