Open unlessdotcom opened 4 years ago
This is the table:
create table factstable
(
visitid varchar(36),
timestamp timestamp,
facts varchar(65535)
);
Nevermind, just read that Amazon announced support exactly for this build into Redshift: https://aws.amazon.com/blogs/aws/new-for-amazon-redshift-data-lake-export-and-federated-queries/ I just tried it and I can export queries directly into S3 formatted as Parquet files.
Thanks for the bug report! So they finally did it :D that's clearly now the best way to perform export from Redshift to Parquet. Probably time for a rethink of if/how Spectrify provides value in light of this functionality!
Is there a recommended tool to do periodic archiving of redshift data to Parquet in S3 and purge the archived data in redshift?
The export command ran fine, it created the zipped files in S3. Next I'm running the convert command, but it exists with this stacktrace:
What does this mean?