search

snowplow / sql-runner

Run templatable playbooks of SQL scripts in series and parallel on Redshift, PostgreSQL, BigQuery and Snowflake
http://snowplowanalytics.com
Apache License 2.0
79 stars 14 forks source link

Release/0.9.2 #172

Closed colmsnowplow closed 3 years ago

colmsnowplow commented 3 years ago

This PR removes the splitting of scripts by ; characters. Instead, we attempt to run the entire script, which can contain multiple statements.

This has the unhappy side effect of causing the BQ api to sometimes fail with success - where any but the last statement hits an error.

I believe this to be a bug in the API, since the query does actually fail, and once the error is retrieved it is of a different format to elsewhere (suggesting that it's potentially been handled for others where it's not handled in this case - will raise an issue with them once I've had a chance to produce a clean repro).

This is handled in our code by explicitly checking for an error after the query is complete, before moving on to the next query.

Additionally, we set Client.Location via the region variable - to mitigate issues which can occur due to BQ sometimes failing to correctly infer region

coveralls commented 3 years ago

Coverage Status

Coverage decreased (-0.2%) to 22.64% when pulling e80a2c9967b589d380b7ec74a7009e8107ba2351 on release/0.9.2 into 71117eca8f8f3a21c163c7acb36f269c3bbec504 on master.

colmsnowplow commented 3 years ago

@jbeemster rc asset confirmed to be all good on prod. Can I get a green tick for release tomorrow?