search

basak / glacier-cli

Command-line interface to Amazon Glacier
Other
617 stars 55 forks source link

Wait hours also for delete? #37

Closed rubendob closed 11 years ago

rubendob commented 11 years ago

Hi

im just wondering what happen when you uses glacier-cli and you request to delete a file .

Do I have to understand also Glacier will wait some hours to retrieve my request and delete the files?

I have done two tests requesting a deletion of a file and then, when I run another time

$ glacier --region eu-west-1 archive list myvault

the file requested to delete it is not showed anymore. so? :)

what happen? You have to wait hours to retrieve a file but the delete is instantaneous ?

This is important to me because I dont want to archive different versions of the same files. So Im thinking to run a script to delete first all uploaded files and then make the upload of the new ones.

Thanks!

basak commented 11 years ago

Short answer: it's complicated. The best thing to do is to version your filenames, say with a sequence number or date incorporated as part of the name.

Long answer:

I'm not sure if the delete is instantaneous; that's up to Amazon. But I do know that the delete will happen, so glacier-cli makes a note of this in its cache, and hides it from you. It needs to do this since Amazon won't actually update its inventory for a while (a day or two), so it needs to remember that it should go away, even if Amazon report that it is present.

However, Amazon don't actually have a concept of a "filename" in Glacier. Just stored objects with arbitrary IDs that are assigned by Amazon at upload time. glacier-cli tracks "filenames" using Amazon's "archive description" field, and Amazon will happily permit multiple archives in the same vault with the same "archive description", so glacier-cli permits this, too. See "Addressing Archives" in the readme.

So: on the same machine using the same glacier-cli cache, deleting an archive and immediately uploading another with the same name should work fine. However, there's a dangerous race condition here. What happens if you encounter a disaster in between deleting the old archive and uploading the new one?

So I can't really advise you here. There are ways around it, but verifying that the process will work safely requires me to understand the details and requirements of your plan. It's just far safer to avoid the issue by using unique filenames, such as by incorporating a version number into them.