Uploads to S3 - Githubissues

LibraryOfCongress / bagger-js

Upload BagIt-format deliveries to S3 entirely in the browser

https://libraryofcongress.github.io/bagger-js/

Other

32 stars 7 forks source link

Uploads to S3 #7

Closed acdha closed 9 years ago

acdha commented 9 years ago

Once we have hashes for a file, we need to upload them. This opens up a number of likely-subtickets, brain-dumped below:

[x] Uploads only start after a file has been hashed so we can do end-to-end fixity checks
[x] Retries for failed uploads – automatic with exponential backoff?
[x] UI for selecting a starting directory
[x] UI to enter S3 credentials
[x] UI to display upload status and performance

acdha commented 9 years ago

A valid bag produced entirely in Chrome, uploaded directly to S3 and passing validation with py-bagit:

screenshot 2015-10-30 12 48 22 2

acdha commented 9 years ago

@eikeon @edsu Do you think the MD5 bits should block this branch merging? It's complicated by the lack of a fast JavaScript MD5 implementation since everyone is deprecating it and since it's transferring over HTTPS and we have a manifest I'm inclined to defer that to a separate ticket.

edsu commented 9 years ago

I definitely don't think MD5 support should block this being integrated. It will still create valid bags, and that's what this tool is for right?!

Awesome work, btw.

acdha commented 9 years ago

@edsu That's what I was thinking – just wanted to run it by someone else to confirm I wasn't being too quick to dismiss it. Thanks!

acdha commented 9 years ago

The AWS S3 servers do not set CORS headers on the global responses, which makes it impossible for us to offer a nice