search

uskudnik / amazon-glacier-cmd-interface

Command line interface for Amazon Glacier
MIT License
374 stars 100 forks source link

Feature idea: automatic download #62

Open wvmarle opened 11 years ago

wvmarle commented 11 years ago

Glacier has a long time lag between requesting archive retrieval and having it available for download. I'm looking for ways to automate this. I'm thinking basically in two directions.

  1. automatic polling. It takes about four hours for an archive to be ready; so when a retrieval request is posted, schedule (using at now + 4 hours) an event to download this file. When run, it checks whether file is available, if so download it, if not at now + 15 minutes try again. Does Windows have such a scheduling tool? Or are Windows users limited to manual polling for their file?
  2. waiting for notification. Set up an SNS notification for this request, have a simple http server running (using Python's CGIHTTPServer module). It seems to be no problem to specify a port so you can run it in parallel to your normal http server, and just have it point to the correct python script behind the http server should do the job of starting glacier-cmd and then starting the download of the archive.

Option 2. is definitely the best; allows for many more actions e.g. ask inventory and have it e-mailed to yourself as soon as it's available.

uskudnik commented 11 years ago

Yeah, I was thinking about option two already but had a feeling that various firewalls and NATs could complicate things for users? Though our users are supposed to be technically literate so it might not be such a problem?

wvmarle commented 11 years ago

Sure - that's anyway for advanced users. Basically: "make sure you have a port open on your machine, and allow outside connections in on that port". How to do that, is beyond the scope of glacier-cmd. I think that people who are interested in this script will be able to manage opening a port.

Probably best to implement both options. It's just that I wouldn't know about a scheduler on Windows! An alternative to a scheduler is of course to let your process "sleep" but then if computer is off in the meantime it breaks. Can't have it all...