search

JeremyGrosser / tablesnap

Uses inotify to monitor Cassandra SSTables and upload them to S3
BSD 2-Clause "Simplified" License
181 stars 86 forks source link

tablesnap crash #64

Closed jwojcik-zz closed 7 years ago

jwojcik-zz commented 9 years ago

My deamonized tablesnap crashed after running for a while.

2015-10-20 03:57:02,873 CRITICAL Failed to lookup keyname after 1 retries Traceback (most recent call last): File "/usr/bin/tablesnap", line 148, in key_exists key = bucket.get_key(keyname) File "/usr/lib/python2.6/site-packages/boto/s3/bucket.py", line 192, in get_key key, resp = self._get_key_internal(key_name, headers, query_args_l) File "/usr/lib/python2.6/site-packages/boto/s3/bucket.py", line 199, in _get_key_internal query_args=query_args) File "/usr/lib/python2.6/site-packages/boto/s3/connection.py", line 664, in make_request retry_handler=retry_handler File "/usr/lib/python2.6/site-packages/boto/connection.py", line 1071, in make_request retry_handler=retry_handler) File "/usr/lib/python2.6/site-packages/boto/connection.py", line 1030, in _mexe raise ex error: [Errno 110] Connection timed out

2015-10-20 03:57:02,874 CRITICAL Failed uploading Aborting. Traceback (most recent call last): File "/usr/bin/tablesnap", line 123, in worker self.upload_sstable(bucket, keyname, f) File "/usr/bin/tablesnap", line 272, in upload_sstable if self.key_exists(bucket, keyname, filename, stat): File "/usr/bin/tablesnap", line 148, in key_exists key = bucket.get_key(keyname) File "/usr/lib/python2.6/site-packages/boto/s3/bucket.py", line 192, in get_key key, resp = self._get_key_internal(key_name, headers, query_args_l) File "/usr/lib/python2.6/site-packages/boto/s3/bucket.py", line 199, in _get_key_internal query_args=query_args) File "/usr/lib/python2.6/site-packages/boto/s3/connection.py", line 664, in make_request retry_handler=retry_handler File "/usr/lib/python2.6/site-packages/boto/connection.py", line 1071, in make_request retry_handler=retry_handler) File "/usr/lib/python2.6/site-packages/boto/connection.py", line 1030, in _mexe raise ex error: [Errno 110] Connection timed out

JeremyGrosser commented 7 years ago

This is expected behavior. I added the --retries option in c9762cb5b0f2379e7325545a362c5002f827fd82, which allows you to configure how many times the upload is retried before failing.