TobikoData / sqlmesh

Efficient data transformation and modeling framework that is backwards compatible with dbt.
https://sqlmesh.com
Apache License 2.0
1.83k stars 161 forks source link

bug report: raise error on dbt source yaml tests field #2538

Closed lei0zhou closed 3 months ago

lei0zhou commented 6 months ago

based on the following dbt source yaml setup, sqlmesh will raise error with Could not render or parse jinja at '/Users/leizhou/Dev/dbt_park/models/raw-sparcs/_sources.yml'. Encountered unknown tag 'test'.

---
version: 2
sources:
  - name: a
    database: abc
    schema: c
    tables:
      - name: b
        tests:
          - dbt_expectations.expect_table_columns_to_contain_set:
              column_list: ['id']
izeigerman commented 6 months ago

Can you please attache a full stacktrace.

lei0zhou commented 6 months ago

thanks! not sure exactly the thing you need. I am attaching the command line output and log file. let me know if you need anything else.

sqlmesh plan
16:45:29  Registered adapter: duckdb=1.7.4
16:45:29  Unable to do partial parsing because a project dependency has been added
Error: Could not render or parse jinja at '/Users/leizhou/Dev/sqlmesh_park/models/example/schema.yml'.
Encountered unknown tag 'test'.
2024-04-30 12:45:29,712 - MainThread - sqlmesh.core.config.connection - INFO - Creating new DuckDB adapter for data files: {'dev.duckdb'} (connection.py:255)
2024-04-30 12:45:29,730 - MainThread - sqlmesh.core.config.connection - INFO - Creating new DuckDB adapter for in-memory database (connection.py:257)
2024-04-30 12:45:30,741 - MainThread - sqlmesh.core.config.connection - INFO - Using existing DuckDB adapter due to overlapping data file: dev.duckdb (connection.py:250)
izeigerman commented 6 months ago

@lei0zhou there should be a file in the logs/ folder which should contain a full backtrace with references to code which caused the exception. If it's not there, can you please try running sqlmesh with the --debug flag to get more details, like:

sqlmesh --debug <command> ...

Thanks!

tobymao commented 3 months ago

not enough info to debug this