It is common for S3 buckets to have SSE enabled, and if the client does not specify the designed SSE option in the ExtraArgs arg of s3.upload_file(), boto3 will give the user a rather non-obvious permissions error.
Analogous to https://github.com/transferwise/pipelinewise-target-redshift/issues/8 (Ability for custom configurations in the COPY statement), a general approach would be to allow the user to supply options for inclusion in the ExtraArgs that is passed to s3.file_upload at db_sync.py#L390:
It is common for S3 buckets to have SSE enabled, and if the client does not specify the designed SSE option in the
ExtraArgsarg ofs3.upload_file(), boto3 will give the user a rather non-obvious permissions error.Analogous to https://github.com/transferwise/pipelinewise-target-redshift/issues/8 (Ability for custom configurations in the COPY statement), a general approach would be to allow the user to supply options for inclusion in the
ExtraArgsthat is passed tos3.file_uploadatdb_sync.py#L390:https://github.com/transferwise/pipelinewise-target-redshift/blob/80796b6c6e4647a85ddc9492e343547a027258e4/target_redshift/db_sync.py#L390
Or at least there should be an option to pass an SSE arg, per https://stackoverflow.com/questions/46610550/how-to-add-encryption-to-boto3-s3-transfer-transferconfig-for-s3-file-upload