Glacier_lib bugs

[wvmarle]

Notes taken while playing around with the glacier_lib branch.

Error not caught:

$ glacier-cmd describevault nonexisting_vault
[...]
 File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/GlacierWrapper.py", line 221, in _check_response
    raise GlacierWrapper.ResponseException("404 Not found " + response.read() + response.msg,
TypeError: cannot concatenate 'str' and 'instance' objects

Not implemented?

$ glacier-cmd listmultiparts Squirrel_backup
[...]
AttributeError: 'GlacierWrapper' object has no attribute 'listmultiparts'

Just used a random string for the ArchiveID as I don't have any test subjects available.

$ glacier-cmd abortmultipart Squirrel_backup werawreawfvqet 
[...]
  File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/GlacierWrapper.py", line 46, in wrapper
    ret = fn(*args, **kwargs)
TypeError: abortmultipart() takes exactly 3 arguments (2 given)

dowload() function is not using prettyprint for its output.

Result of an attempt to delete an archive that was deleted already (archive ID removed for readability)

$ glacier-cmd rmarchive Squirrel_backup <archiveID>
204 No Content
Traceback (most recent call last):
  File "/usr/local/bin/glacier-cmd", line 9, in <module>
    load_entry_point('glacier==0.2dev', 'console_scripts', 'glacier-cmd')()
  File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/glacier.py", line 804, in main
    args.func(args)
  File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/glacier.py", line 449, in rmarchive
    parse_response( gv.delete_archive(archive) )
  File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/glacier.py", line 95, in parse_response
    print_headers(response)
  File "/usr/local/lib/python2.7/dist-packages/glacier-0.2dev-py2.7.egg/glacier/glacier.py", line 79, in print_headers
    for header in headers:
TypeError: iteration over non-sequence

inventory should check whether the are any jobs "in progress", if so inform the user of this situation and not create yet another inventory retrieval job as it does now.

My intended activity in this library:

• add logging, lots of it, multiple levels. The logging service should be in good shape now.
• move all the output (now using prettyprint for tables) to a new class GlacierOutput which can handle all kinds of output. Will start with just the prettyprint as it's now - this should be the default if run from TTY. Have to come up with a proper interface to pass the data to this class and how to handle the passing of arguments for the required export format(s).

It seems fairly complete, but I didn't really run tests on it.

[wvmarle]

Glacier_lib branch seems to be quite OK for now.

Search is not working; the rest is. Some functions renamed, others repurposed - in a way that makes a bit more sense to me. Names may have made sense at first of course.

getarchive now starts the archive retrieval job. Does not do searching - a front-end has to call search separately, then select the archive id for retrieval.

download now does the actual downloading of the file - but no block download or so yet.

search is broken.

I'm using this branch now on my server for nightly uploads of archives from bacula to glacier, seems to work.

usage: glacier-cmd [-h] [-c FILE] [--logtostdout]
                   [--aws-access-key AWS_ACCESS_KEY]
                   [--aws-secret-key AWS_SECRET_KEY] [--region REGION]
                   [--bookkeeping]
                   [--bookkeeping-domain-name BOOKKEEPING_DOMAIN_NAME]
                   [--logfile LOGFILE]
                   [--loglevel {-1,DEBUG,0,INFO,1,WARNING,2,ERROR,3,CRITICAL}]

                   {mkvault,lsvault,describevault,rmvault,upload,listmultiparts,abortmultipart,inventory,getarchive,download,rmarchive,search,listjobs,describejob}
                   ...

Command line interface for Amazon Glacier

optional arguments:
  -h, --help            show this help message and exit
  -c FILE, --conf FILE  Name of the file to log messages to. (default:
                        ~/.glacier-cmd)
  --logtostdout         Send log messages to stdout instead of the config
                        file. (default: False)

Subcommands:
  {mkvault,lsvault,describevault,rmvault,upload,listmultiparts,abortmultipart,inventory,getarchive,download,rmarchive,search,listjobs,describejob}
                        For subcommand help, use: glacier-cmd <subcommand> -h
    mkvault             Create a new vault.
    lsvault             List available vaults.
    describevault       Describe a vault.
    rmvault             Remove a vault.
    upload              Upload an archive to Amazon Glacier.
    listmultiparts      List all active multipart uploads.
    abortmultipart      Abort a multipart upload.
    inventory           List inventory of a vault, if available. If not
                        available, creates inventory retrieval job if none
                        running already.
    getarchive          Requests to make an archive available for download.
    download            Download a file by archive id.
    rmarchive           Remove archive from Amazon Glacier.
    search              Search Amazon SimpleDB database for available archives
                        (requires bookkeeping to be enabled).
    listjobs            List active jobs in a vault.
    describejob         Describe a job.

aws:
  --aws-access-key AWS_ACCESS_KEY
                        Your aws access key (Required if you have not created
                        a ~/.glacier-cmd or /etc/glacier-cmd.conf config file)
                        (default: AKIAIP5VPUSCSJQ6BSSQ)
  --aws-secret-key AWS_SECRET_KEY
                        Your aws secret key (Required if you have not created
                        a ~/.glacier-cmd or /etc/glacier-cmd.conf config file)
                        (default: WDgq6ZZn7Y4Lkt5LxPuionw2pTLbonwdFZz1BGtS)

glacier:
  --region REGION       Region where you want to store your archives (Required
                        if you have not created a ~/.glacier-cmd or /etc
                        /glacier-cmd.conf config file) (default: us-east-1)
  --bookkeeping         Should we keep book of all created archives. This
                        requires a Amazon SimpleDB account and its bookkeeping
                        domain name set (default: True)
  --bookkeeping-domain-name BOOKKEEPING_DOMAIN_NAME
                        Amazon SimpleDB domain name for bookkeeping. (default:
                        squirrel)
  --logfile LOGFILE     File to write log messages to. (default: /home/wouter
                        /.glacier-cmd.log)
  --loglevel {-1,DEBUG,0,INFO,1,WARNING,2,ERROR,3,CRITICAL}
                        Set the lowest level of messages you want to log.
                        (default: DEBUG)

[wvmarle]

All resolved; closing.