I've just setup the snapshotter, and when I run the command, it compresses and seemingly transfers everything to S3 (looks like it's all there), but then it errors at the end with the nodetool ring command as follows:
[hostname] run: /usr/bin/nodetool ring
Traceback (most recent call last):
File "/usr/local/bin/cassandra-snapshotter", line 9, in <module>
load_entry_point('cassandra-snapshotter==0.5.0', 'console_scripts', 'cassandra-snapshotter')()
File "/usr/local/lib/python2.7/site-packages/cassandra_snapshotter/main.py", line 280, in main
run_backup(args)
File "/usr/local/lib/python2.7/site-packages/cassandra_snapshotter/main.py", line 76, in run_backup
worker.snapshot(snapshot)
File "/usr/local/lib/python2.7/site-packages/cassandra_snapshotter/snapshotting.py", line 351, in snapshot
self.write_ring_description(snapshot)
File "/usr/local/lib/python2.7/site-packages/cassandra_snapshotter/snapshotting.py", line 400, in write_ring_description
self.write_on_S3(snapshot.s3_bucket, ring_path, content)
File "/usr/local/lib/python2.7/site-packages/cassandra_snapshotter/snapshotting.py", line 394, in write_on_S3
key.set_contents_from_string(content)
File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1426, in set_contents_from_string
encrypt_key=encrypt_key)
File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 1293, in set_contents_from_file
chunked_transfer=chunked_transfer, size=size)
File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 750, in send_file
chunked_transfer=chunked_transfer, size=size)
File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 951, in _send_file_internal
query_args=query_args
File "/usr/lib/python2.7/dist-packages/boto/s3/connection.py", line 665, in make_request
retry_handler=retry_handler
File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 1071, in make_request
retry_handler=retry_handler)
File "/usr/lib/python2.7/dist-packages/boto/connection.py", line 940, in _mexe
request.body, request.headers)
File "/usr/lib/python2.7/dist-packages/boto/s3/key.py", line 884, in sender
response.status, response.reason, body)
boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
<?xml version="1.0" encoding="UTF-8"?>
<Error><Code>AccessDenied</Code><Message>Access Denied</Message><RequestId>6C23CB43B06B5909</RequestId><HostId>z6yqnOVBCfLjnF0xTB1jyZaKpsVd4Q+Vgizvl9iJCxbgH573MFXuu9UCfCKlv1nvn2Fp/Ronxlo=</HostId></Error>
I've just discovered that this is happening because of encryption. Without the --s3-ssenc flag and with the bucket encryption policy removed, everything completes. Any known reason why the nodetool ring fails when encryption is enabled?
I've just setup the snapshotter, and when I run the command, it compresses and seemingly transfers everything to S3 (looks like it's all there), but then it errors at the end with the
nodetool ring
command as follows:I've just discovered that this is happening because of encryption. Without the
--s3-ssenc
flag and with the bucket encryption policy removed, everything completes. Any known reason why thenodetool ring
fails when encryption is enabled?S3 bucket policy