We are seeing an issue when we have a huge set of data in snowflake,
where we are calculating the stddev and variance on few number columns.
While doing so, its resulting into a 34 digit number, which snowfalke is unable to support and its throwing an error.
As a result the whole query is getting failed. Currently i see that all metrics are captured in soda using a single query and due to this if one metric fails the entire query is failing.Though with a single query optimization is acheived , flexibility and isolation of the issue are missing. With 1 expected/would be metric failure we dont want to loose results and tests on others when soda scan happens.
Providing configuration option to specific columns would be a good option here.
Hope we have some configuration sections like isolated_columns where columns which may bubble up an exception can be mentioned
and a seperate query is created for them from soda
We are seeing an issue when we have a huge set of data in snowflake, where we are calculating the stddev and variance on few number columns. While doing so, its resulting into a 34 digit number, which snowfalke is unable to support and its throwing an error. As a result the whole query is getting failed. Currently i see that all metrics are captured in soda using a single query and due to this if one metric fails the entire query is failing.Though with a single query optimization is acheived , flexibility and isolation of the issue are missing. With 1 expected/would be metric failure we dont want to loose results and tests on others when soda scan happens. Providing configuration option to specific columns would be a good option here. Hope we have some configuration sections like isolated_columns where columns which may bubble up an exception can be mentioned and a seperate query is created for them from soda