Closed Fokko closed 1 year ago
Thank you for opening this issue and the associated PR @Fokko! 🏆
Thank you for opening this issue, fixing of this issue will be helpful also to the company I work in. As you said, this issue happens in session type connection and not in thrift because the exception_handler in connections.py wraps the exception only in case of Thrift. However, I think the solution in the PR related to this issue is good.
Just curious, SHOW TABLE EXTENDED is not supported for v2 tables.;
this is something everyone facing, right?
SHOW TABLE EXTENDED is not supported for v2 tables.;
This will be resolved in Spark v4: https://github.com/apache/spark/commit/c91dbde2f94c15d2007dccaeec5d72a159e9f4e2
Is this a new bug in dbt-spark?
Current Behavior
When you run a
session
type connection (where Spark runs in the same Python process), when an error is made, aAnalysisException
will be thrown. This isn't the case with a Thrift connection, and it does break the logic to catch exception indbt-spark
.Expected Behavior
When a SQL statement fails, an
AnalysisException
should be thrown.Steps To Reproduce
dbt run
Relevant log output
Additional Context
No response