Incremental backups fail when manifest.json not in root of bucket prefix

Given an S3 bucket with paths like cassandra-backups/staging/20141211231905... this line returns an array like [u'staging/', u'staging/20141211231905/']. The get_contents_as_string() call then tries to look up a path like cassandra-backups/staging/manifest.json which of course fails with a 404.

Here's a backtrace:

Traceback (most recent call last):
  File "/usr/bin/cassandra-snapshotter", line 9, in <module>
    load_entry_point('cassandra-snapshotter==0.3.0', 'console_scripts', 'cassandra-snapshotter')()
  File "/usr/lib/python2.6/site-packages/cassandra_snapshotter-0.3.0-py2.6.egg/cassandra_snapshotter/main.py", line 198, in main
    run_backup(args)
  File "/usr/lib/python2.6/site-packages/cassandra_snapshotter-0.3.0-py2.6.egg/cassandra_snapshotter/main.py", line 38, in run_backup
    table=args.table
  File "/usr/lib/python2.6/site-packages/cassandra_snapshotter-0.3.0-py2.6.egg/cassandra_snapshotter/snapshotting.py", line 435, in get_snapshot_for
    for snapshot in self:
  File "/usr/lib/python2.6/site-packages/cassandra_snapshotter-0.3.0-py2.6.egg/cassandra_snapshotter/snapshotting.py", line 445, in __iter__
    self._read_s3()
  File "/usr/lib/python2.6/site-packages/cassandra_snapshotter-0.3.0-py2.6.egg/cassandra_snapshotter/snapshotting.py", line 418, in _read_s3
    manifest_data = mkey.get_contents_as_string()
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 1780, in get_contents_as_string
    response_headers=response_headers)
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 1648, in get_contents_to_file
    response_headers=response_headers)
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 1480, in get_file
    query_args=None)
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 1512, in _get_file_internal
    override_num_retries=override_num_retries)
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 343, in open
    override_num_retries=override_num_retries)
  File "/usr/lib/python2.6/site-packages/boto-2.34.0-py2.6.egg/boto/s3/key.py", line 303, in open_read
    self.resp.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 404 Not Found
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>NoSuchKey</Code><Message>The specified key does not exist.</Message><Key>cassandra-backups/manifest.json</Key><RequestId>A777F43BA4BCC13F</RequestId><HostId>/qU27mGG5n7cPd4KP8UfcaiwTjGNzCP58HakJqYVhIn6KFWBoS6ZBps4pSj7btI/</HostId></Error>

tbarbugli / cassandra_snapshotter

Incremental backups fail when manifest.json not in root of bucket prefix #36