This pull request adds a fivetran_s3_role module, which allows Fivetran to access buckets on our account.
How should this be reviewed?
This is based on the Fivetran S3 Setup Guide here, and follows the example set by @blisteringherb's work in #250 (to add a similar connection for CloudWatch logs). Since we want to give Fivetran access to only a few specific buckets, I've opted to add this as a resource that we attach to any buckets that need it (rather than a standalone "global" role).
I'll try creating a connector on development & QA (which have relatively few clicks) before doing so on production.
Any background context you want to provide?
This will allow us to replicate this private S3 bucket (where Bertly 2.0 stores click information) into our data warehouse with Fivetran. We're storing clicks here, instead of PostgreSQL, to save money & hopefully reduce complexity.
What's this PR do?
This pull request adds a
fivetran_s3_rolemodule, which allows Fivetran to access buckets on our account.How should this be reviewed?
This is based on the Fivetran S3 Setup Guide here, and follows the example set by @blisteringherb's work in #250 (to add a similar connection for CloudWatch logs). Since we want to give Fivetran access to only a few specific buckets, I've opted to add this as a resource that we attach to any buckets that need it (rather than a standalone "global" role).
I'll try creating a connector on development & QA (which have relatively few clicks) before doing so on production.
Any background context you want to provide?
This will allow us to replicate this private S3 bucket (where Bertly 2.0 stores click information) into our data warehouse with Fivetran. We're storing clicks here, instead of PostgreSQL, to save money & hopefully reduce complexity.
Relevant tickets
References Pivotal #172652716.
Checklist