search

TobikoData / sqlmesh

Efficient data transformation and modeling framework that is backwards compatible with dbt.
https://sqlmesh.com
Apache License 2.0
1.84k stars 161 forks source link

tracebacks sometimes show in failing tests #3355

Closed esciara closed 2 weeks ago

esciara commented 2 weeks ago

I open this issue to trace the fix for the problem, first mentioned in https://github.com/TobikoData/sqlmesh/issues/3353#issuecomment-2466291702. Ping @georgesittas .

sqlmesh --version
0.131.0

As a repeat of the results in #3352, here is an example with no traceback:

sqlmesh test
...F......
======================================================================
FAIL: test_arrays_bug (/project-dir/tests/test_arrays_bug.yaml)
----------------------------------------------------------------------
AssertionError: Data mismatch (exp: expected, act: actual)

  aggregated_duplicates        
                    exp     act
0                (c, b)  (b, c)

----------------------------------------------------------------------
Ran 10 tests in 0.159s

FAILED (failures=1)

but for this one (with my initially wrong test), there is a traceback:

sqlmesh test
........F..
======================================================================
FAIL: reproduce_bug (/project-dir/sqlmesh-20-duplicate-arbitrage/tests/test_recursive_cte_bug.yaml) (cte='"id_contact_a_with_aggregated_id_contact_bs"')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "project-dir/sqlmesh/core/test/definition.py", line 550, in test_ctes
    self.assert_equal(expected, actual, sort=sort, partial=partial)
  File "project-dir/sqlmesh/core/test/definition.py", line 273, in assert_equal
    raise e
  File "project-dir/sqlmesh/core/test/definition.py", line 247, in assert_equal
    pd.testing.assert_frame_equal(
  File "project-dir/.venv/lib/python3.12/site-packages/pandas/_testing/asserters.py", line 1218, in assert_frame_equal
    raise_assert_detail(
  File "project-dir/.venv/lib/python3.12/site-packages/pandas/_testing/asserters.py", line 614, in raise_assert_detail
    raise AssertionError(msg)
AssertionError: Data mismatch (rows are different)

Unexpected rows:

  id_contact_a aggregated_id_contact_bs
0            b             (c, d, e, f)
1            c                   (d, f)

----------------------------------------------------------------------
Ran 11 tests in 0.176s

FAILED (failures=1)
georgesittas commented 2 weeks ago

I fixed this in the other PR I merged, it should work in main.