search

bbc / digital-paper-edit-client

Work in progress - BBC News Labs digital paper edit project - React Client
https://bbc.github.io/digital-paper-edit-client
Other
45 stars 13 forks source link

S3 file upload cloud formation setup #35

Open emettely opened 5 years ago

emettely commented 5 years ago

Context

We want to upload directly to S3 from the client using the signed URL.

Acceptance Criteria

-Make sure upload of files are only done through signed URLs, retrieved from the API- (moved to a different ticket)

Some useful references

emettely commented 5 years ago

I'm setting up the S3 as part of the API.

emettely commented 5 years ago

The S3 is setup for int and test bundled with the API. The permissions from the API is:

                                "s3:ListBucket",
                                "s3:*Object"

https://github.com/bbc/digital-paper-edit-infrastructure/blob/master/dpe-api/infrastructure/src/main.py

^ This is the bit that has that logic. @jamesdools and @pietrop , we might want to consider what the life-cycle (expire in 1 year or half a year...) of items in the bucket should be. I have it currently commented out - but once we figure out the subdirectory names (e.g. audio, original, preview etc.) we can uncomment + implement the lifecycle policy.

pietrop commented 5 years ago

@emettely can you add some more details on how this works?

I get there's something added to the cloud formation to give the EC2 instance for the API serve the capability to access the bucket(?)

Do we know more about how do you then access (add or get files from the S3 bucket) from within the express server inside the API? (do we use the AWS Node SDK?)

Also I see the link goes straight into master, was there a PR for this?

And is this still in progress ready for review or done?

pietrop commented 5 years ago

narrowing down this card to be only covering first part of acceptance criteria to do with

moving the rest to a separate card https://github.com/bbc/digital-paper-edit-api/issues/4

pietrop commented 5 years ago

To review this (draft)

  1. Use python script to generates cloud formation file
  2. Use Jenkins to upload to AWS (stack job)
  3. see if it gets picked up in int, test etc.. in cosmos
  4. in AWS console you should see the resources with right policies
emettely commented 5 years ago

https://github.com/bbc/digital-paper-edit-infrastructure/compare/b04120ea8a865557946bfebfe851578e77833212...c33727bfbe9a3afc197356db61c87c55e97c5679