search

hellonarrativ / spectrify

Export Redshift data and convert to Parquet for use with Redshift Spectrum or other data warehouses.
https://aws.amazon.com/blogs/big-data/narrativ-is-helping-producers-monetize-their-digital-content-with-amazon-redshift/
MIT License
116 stars 25 forks source link

Adding REGION parameter to UNLOAD #49

Closed xanderayes closed 5 years ago

xanderayes commented 5 years ago

The region parameter is necessary for the unload function. If it's not there, S3 raises an error: "S3ServiceException:The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.,Status 301,Error PermanentRedirect".

Please keep in mind that my pull request is just an example. "us-east-1" should not be hard coded. It should be asked as a parameter beforehand.

poberherr commented 5 years ago

why would you hardcode the region ?

It could be a default which is assumed when no other input is given.

xanderayes commented 5 years ago

why would you hardcode the region ?

It could be a default which is assumed when no other input is given.

Your suggestion is perfect. I just hard-coded it to clarify what the problem is. I'm not actually interested in changing the code myself. It's just an issue that I'm facing when trying to use this library. Maybe I should have posted this as an issue instead.

c-nichols commented 5 years ago

Closing for now since it’s been a while (let me know if you’re still working on this)