uskudnik / amazon-glacier-cmd-interface

Command line interface for Amazon Glacier
MIT License
375 stars 103 forks source link

Issues working with more than 50 archive downloads #140

Closed zg closed 10 years ago

zg commented 10 years ago

I am having an issue with the getarchive function. After putting in 50 jobs, I cannot seem to get around this error:

Traceback (most recent call last):
  File "/usr/bin/glacier-cmd", line 9, in <module>
    load_entry_point('glacier==0.2dev', 'console_scripts', 'glacier-cmd')()
  File "/usr/lib/python2.7/site-packages/glacier/glacier.py", line 929, in main
    args.func(args)
  File "/usr/lib/python2.7/site-packages/glacier/glacier.py", line 156, in wrapper
    return fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/glacier.py", line 343, in getarchive
    status, job, jobid = glacier.getarchive(args.vault, args.archive)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 65, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 231, in glacier_connect_wrap
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 65, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 1278, in getarchive
    job_list = self.list_jobs(vault_name)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 65, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 231, in glacier_connect_wrap
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 65, in wrapper
    ret = fn(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/glacier/GlacierWrapper.py", line 783, in list_jobs
    marker=marker)
  File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 288, in list_jobs
    return self.make_request('GET', uri, params=params)
  File "/usr/lib/python2.7/site-packages/boto/glacier/layer1.py", line 77, in make_request
    for key, value in params:
ValueError: too many values to unpack

And the command I'm attempting to run:

glacier-cmd --aws-access-key ACCESSKEY --aws-secret-key SECRETKEY --region us-east-1 getarchive ARCHIVEID
wvmarle commented 10 years ago

Does your archive ID by chance start with a - (hyphen) character? If so add the extra switch -- in front of it, as otherwise it's interpreted as a command line switch instead. The -- tells the script that there are no more switches, so a value starting with - won't be misinterpreted.

zg commented 10 years ago

No, but I did prepend -- to the beginning of every archive ID that starts with a dash (-).

zg commented 10 years ago

I ran through the script again, with the same issue occurring again. What happened this time was that at the 51st job, the program encountered the error I pasted above. Is there some kind of limitation that is established by the program in terms of the number of jobs that it can handle? Or perhaps Amazon Glacier has a limit to the number of jobs that you can have.

zg commented 10 years ago

Unrelated to the code. This was an issue with ArchLinux AUR package "boto-glacier-git" not using the latest version of boto.