snowplow / snowplow-rdb-loader

Stores Snowplow enriched events in Redshift, Snowflake and Databricks
Other
31 stars 17 forks source link

Allow for setting wlm_query_slot_count for Redshift imports #422

Open smugryan opened 9 years ago

smugryan commented 9 years ago

Redshift/postgresql has massive amounts of knobs to turn for optimizing query performance and one recommended by AWS is increasing wlm_query_slot_count in a session before running a query that needs extra CPU and memory.

Docs: http://docs.aws.amazon.com/redshift/latest/dg/cm-c-defining-query-queues.html

Would be awesome to allow users to set this in the redshift config. We saw massive (10x faster vacuums) improvements when increasing ours to 45, but we need to do that manually for now.

alexanderdean commented 9 years ago

Nice idea!