search

hellonarrativ / spectrify

Export Redshift data and convert to Parquet for use with Redshift Spectrum or other data warehouses.
https://aws.amazon.com/blogs/big-data/narrativ-is-helping-producers-monetize-their-digital-content-with-amazon-redshift/
MIT License
116 stars 25 forks source link

How can I simply import s3 folder (no Spectrum ) #44

Closed eromoe closed 5 years ago

eromoe commented 5 years ago

Hello,

The official copy command would got error, so I look into this project .

I see usage is

Usage: spectrify create_table [OPTIONS] S3_PATH SOURCE_TABLE DEST_TABLE

Here, what is SOURCE_TABLE ?? I already have a s3 folder contains some parquet files . And , redshift spectrum is not launched on China AWS .

I just want to import data from s3 to redshift .

c-nichols commented 5 years ago

Hi @eromoe this article may help — https://aws.amazon.com/about-aws/whats-new/2018/06/amazon-redshift-can-now-copy-from-parquet-and-orc-file-formats/

I do not think you need Spectrify to accomplish your goals - if you have any other questions the aws forums are also super helpful.