Textualize / pytest-textual-snapshot

Snapshot testing for Textual applications
MIT License
21 stars 4 forks source link

Exits with failure code when test xfails #6

Closed tconbeer closed 11 months ago

tconbeer commented 12 months ago

If you mark a snapshot test xfail and it fails, pytest exits with a nonzero code. For example.

============================= test session starts ==============================
platform linux -- Python 3.11.5, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/runner/work/textual-fastdatatable/textual-fastdatatable
plugins: asyncio-0.21.1, textual-snapshot-0.4.0, syrupy-3.0.6
asyncio: mode=Mode.STRICT
collected 23 items

tests/snapshot_tests/test_snapshots.py ......sx.ss                       [ 47%]
tests/unit_tests/test_arrow_backend.py ............                      [100%]

--------------------------- snapshot report summary ----------------------------
1 snapshot failed. 7 snapshots passed. 3 snapshots unused.

Re-run pytest with --snapshot-update to delete unused snapshots.
Textual Snapshot Report

1 mismatched snapshots

View the failure report.

/home/runner/work/textual-fastdatatable/textual-fastdatatable/snapshot_report.ht
ml

=================== [19](https://github.com/tconbeer/textual-fastdatatable/actions/runs/6343838093/job/17232612976?pr=1#step:8:20) passed, 3 skipped, 1 xfailed in 4.13s ===================
Error: Process completed with exit code 1.

The test is:

@pytest.mark.xfail(reason="...")
def test_datatable_style_ordering(snap_compare) -> None:
    # Regression test for https -> None://github.com/Textualize/textual/issues/2061
    assert snap_compare(SNAPSHOT_APPS_DIR / "data_table_style_order.py")
tconbeer commented 11 months ago

There is a syrupy cli option that fixes this: --snapshot-warn-unused