Refactor list relation macros to use sys catalog instead of information schema causing concurrency issues when running multiple threads in parallel (https://github.com/microsoft/dbt-fabric/issues/52).
This change allows us to run our integration tests but from occasionally there are still locks.
E pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Snapshot isolation transaction failed in database 'dbtfabricadapter' because the object accessed by the statement has been modified by a DDL statement in another concurrent transaction since the start of this transaction. It is disallowed because the metadata is not versioned. A concurrent update to metadata can lead to inconsistency if mixed with snapshot isolation. (3961) (SQLExecDirectW)")
Fabric only support Snapshot isolation level, so the "trick" of turning off and on again is not going to work i think...
Also recommended to set too non-isolation before commit for SQL Server official docs here.
TLDR, for Fabric we have a limitation of having only one isolation level which imo can't be changed, nor switched off with SET TRANSACTION hints so at this point i see no workaround until Fabric starts to support other isolation types.
Enhancements
Decouple imports to common dbt core and dbt adapter interface packages for future maintainability and extensibility.
Bump adapter packages
from pyodbc>=4.0.35,<5.1.0" to pyodbc>=4.0.35,<5.2.0
From now on, Apple-silicon users don't have to locally build pyodbc, because M1, M2 binaries is included in pyodbc from 5.1.0 onwards!
Features
Supporting dbt-core 1.8.0
Bug fixes
This change allows us to run our integration tests but from occasionally there are still locks.
Fabric only support Snapshot isolation level, so the "trick" of turning off and on again is not going to work i think...
couple refs:
TLDR, for Fabric we have a limitation of having only one isolation level which imo can't be changed, nor switched off with
SET TRANSACTIONhints so at this point i see no workaround until Fabric starts to support other isolation types.Enhancements
Decouple imports to common dbt core and dbt adapter interface packages for future maintainability and extensibility.