File "removeVault.py", line 81, in <module>

[r3cgm]

Thank you for writing this tool. I'm trying to remove a couple old crufty glacier archives. After trying to write my own boto utility I stumbled on yours. It is throwing the following error however. Any ideas what I can do to fix this?

INFO : Logging level set to DEBUG. INFO : Connecting to Amazon Glacier... INFO : Getting selected vault... INFO : Getting jobs list... INFO : Found existing inventory retrieval job... INFO : Inventory retrieved, parsing data... Traceback (most recent call last): File "removeVault.py", line 81, in inventory = json.loads(job.get_output().read()) File "/Library/Python/2.7/site-packages/boto/glacier/job.py", line 89, in get_output byte_range) File "/Library/Python/2.7/site-packages/boto/glacier/layer1.py", line 740, in get_job_output response_headers=response_headers) File "/Library/Python/2.7/site-packages/boto/glacier/layer1.py", line 116, in make_request return GlacierResponse(response, response_headers) File "/Library/Python/2.7/site-packages/boto/glacier/response.py", line 40, in init body = json.loads(http_response.read().decode('utf-8')) File "/Library/Python/2.7/site-packages/boto/connection.py", line 410, in read self._cached_response = http_client.HTTPResponse.read(self) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 551, in read s = self._safe_read(self.length) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 658, in _safe_read chunk = self.fp.read(min(amt, MAXAMOUNT)) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 380, in read data = self._sock.recv(left) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 241, in recv return self.read(buflen) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 160, in read return self._sslobj.read(len) ssl.SSLError: The read operation timed out

[leeroybrun]

I'm happy if it can help you, thanks for the message !

Which version of boto is installed on your system ?

It can maybe be a network issue, as the last line suggest : ssl.SSLError: The read operation timed out

Have you tried running it on a different machine/network ? Are you on your personal network or a public one ?

[r3cgm]

Thanks for the help!

I'm running boto 2.34.0.

I don't have another machine set up to run this yet but I could try that.

I'm pretty sure the network is normal. I have a vanilla connection on Sonic.net in the San Francisco Bay Area and they are good about not interfering or filtering traffic.

This is running from my personal (NAT'ed) network at home, against AWS/Glacier.

I'm not sure if this is a clue or not, but it took a looooong time to do this in my script before I tried switching over to your utility: glacier_layer_1.get_job_output('myvault', 'my job')

Maybe there's some timeout baked into the code here? I think it might have taken more than 5 minutes to do the manual job dump.

Other than that, it does seem like your glacier-vault-remove...

LOL, it just started working! I didn't change anything but now it is doing this:

INFO : Removing archives... please be patient, this may take some time...

So I guess maybe a false alarm here. Weird.

[leeroybrun]

Great ! Maybe a temporary error, Glacier can be a bit unpredictable sometimes. :-)

The archives' removal can take a while, keep me updated about the success/failure !

Le vendredi 28 novembre 2014, Christopher Mann notifications@github.com a écrit :

Thanks for the help!

I'm running boto 2.34.0.

I don't have another machine set up to run this yet but I could try that.

I'm pretty sure the network is normal. I have a vanilla connection on Sonic.net in the San Francisco Bay Area and they are good about not interfering or filtering traffic.

This is running from my personal (NAT'ed) network at home, against AWS/Glacier.

I'm not sure if this is a clue or not, but it took a looooong time to do this in my script before I tried switching over to your utility: glacier_layer_1.get_job_output('myvault', 'my job')

Maybe there's some timeout baked into the code here? I think it might have taken more than 5 minutes to do the manual job dump.

Other than that, it does seem like your glacier-vault-remove...

LOL, it just started working! I didn't change anything but now it is doing this:

INFO : Removing archives... please be patient, this may take some time...

So I guess maybe a false alarm here. Weird.

— Reply to this email directly or view it on GitHub https://github.com/leeroybrun/glacier-vault-remove/issues/4#issuecomment-64921207 .

[r3cgm]

Will do. I have 2 glacier archives about 300 gigs each so we'll see how long this takes. My main problem now is to figure out how to get OSX Yosemite from suspending the network and going to sleep, otherwise the script will terminate. :)

[r3cgm]

Well, it took about 2 days to clean out one archive and 3 days for the other. I am happy to report that glacier-vault-remove worked perfectly during that time.

The only minor feedback I might suggest is that during normal output a lot of these get generated:

ERROR : [Errno 36] Operation now in progress

I think those are ignorable? I mean, those are just status information "hang on, the script is still running" at least is how I am interpreted this.

Again, thanks for writing this tool!

[leeroybrun]

Fine if it worked ! I'm glad to read this :-)

Do you maybe have a line number for the error ? Seems to be thrown when we have a non-blocking socket and the connection cannot be completed immediately.

[r3cgm]

I'm pretty sure it's line 90, though the error seems to be indirect:

            except Exception, e:                                                   
                **logging.error(e)**

                logging.info('Sleep 2 mins before retrying...')                    
                time.sleep(60*2)                                                   

                logging.info('Retry to remove archive ID : %s', archive['ArchiveId'])
                try:                                                               
                    vault.delete_archive(archive['ArchiveId'])                     
                    logging.info('Successfully removed archive ID : %s', archive['ArchiveId'])
                except:                                                            
                    logging.error('Cannot remove archive ID : %s', archive['ArchiveId'])

It would do this same cycle every time:

INFO : Retry to remove archive ID : xfQjsBLSnS1mvec... INFO : Successfully removed archive ID : xfQjsBLSnS1mvec... ERROR : [Errno 36] Operation now in progress INFO : Sleep 2 mins before retrying... INFO : Retry to remove archive ID : r_akZ0mqp... INFO : Successfully removed archive ID : r_akZ0mqp... ERROR : [Errno 36] Operation now in progress INFO : Sleep 2 mins before retrying...