search

thoughtspot / cs_tools

Scale your ThoughtSpot adoption with tools created by the ThoughtSpot Solutions Consulting organization.
https://thoughtspot.github.io/cs_tools/
Other
8 stars 6 forks source link

created redshift syncer #121

Closed saurabhsingh1608 closed 5 months ago

saurabhsingh1608 commented 5 months ago

Created new redshift syncer from scratch and used redshift-connector and sqlalchemy to connect to redshift. To use this connector we need to make VPC public

boonhapus commented 5 months ago

@saurabhsingh1608 are you able to share Redshift documentation links or process for making VPC public?

I'd like to include any setup steps in a section at the top like we have for Google based syncers

boonhapus commented 5 months ago

Can you also link me to Table.merge documentation for UPSERT capability? This would be a great capability, but I want to review it.

boonhapus commented 5 months ago

Is there a default port? Usually one of 8080 , 8989 , or 443 .

saurabhsingh1608 commented 5 months ago
  1. For making VPC public ( we can add GIF also ) -> https://repost.aws/knowledge-center/redshift-cluster-private-public.
  2. For merge command ( i need to test it more because there is one more command we can use ) -> https://docs.aws.amazon.com/redshift/latest/dg/r_MERGE.html
  3. Default port is 5439 -> https://docs.aws.amazon.com/redshift/latest/mgmt/serverless-connecting.html
boonhapus commented 5 months ago

This is great @saurabhsingh1608 , thanks! For 2/ I will replace it with the generic upsert while we investigate MERGE INTO in the underlying driver. This can be a good performance improvement as we've seen in Snowflake's case.