[Bug]: Concurrency issue in Snowflake when running upload_results macro

[lucho-chavez]

Overview

Getting the following error when running multiple pipelines in parallel and finishing almost at the same time ( Event driven ingestion pipelines)

000625 (57014): Statement '01b43b22-0509-2fce-0003-29019743c0af' has locked table 'MODELS' in transaction 1715330074481000000 and this lock has not yet been released. 08:34:38 Your statement '01b43b22-0509-2fce-0003-29019743c13b' was aborted because the number of waiters for this lock exceeds the 20 statements limit.

How to reproduce

Run multiple pipelines at the same time and finishing almost at the same time.

Expected behaviour

No concurrency issues when running multiple pipelines in parallel all invoking "{{ dbt_artifacts.upload_results(results) }}"

Screenshots

Environment

Results of running dbt --version: 1.5

08:34:27  Uploading snapshot executions
08:34:27  Uploading test executions
08:34:27  Uploading exposures
08:34:29  Uploading tests
08:34:32  Uploading seeds
08:34:32  Uploading models
08:34:37  Database error while running on-run-end
08:34:37  
08:34:37
08:34:37  Finished running 1 view model in 0 hours 0 minutes and 21.17 seconds (21.17s).
08:34:38  
08:34:38
08:34:38  Completed with 1 error and 0 warnings:
08:34:38  
08:34:38  on-run-end failed, error:
08:34:38   000625 (57014): Statement '01b43b22-0509-2fce-0003-29019743c0af' has locked table 'MODELS' in transaction 1715330074481000000 and this lock has not yet been released.
08:34:38  Your statement '01b43b22-0509-2fce-0003-29019743c13b' was aborted because the number of waiters for this lock exceeds the 20 statements limit.
08:34:38  
08:34:38
08:34:38  Done. PASS=1 WARN=0 ERROR=1 SKIP=0 TOTAL=2

Please paste the contents of your packages.yml file here:

packages:
 - package: calogica/dbt_expectations
    version: [">=0.8.0", "<0.9.0"]
  - package: brooklyn-data/dbt_artifacts
    version: [">=2.4.0", "<2.5.0"]
  - package: dbt-labs/metrics
    version: [">=1.5.0", "<1.6.0"] 
  - package: dbt-labs/codegen
    version: 0.10.0
  ...