15r10nk / inline-snapshot

create and update inline snapshots in your python tests
https://15r10nk.github.io/inline-snapshot/
MIT License
343 stars 10 forks source link

"N snapshots can be trimmed" but nothing is trimmed? #78

Open pawamoy opened 2 months ago

pawamoy commented 2 months ago

When I run my test suite, I get this output at the end:

=============== inline snapshot ====================
Info: 11 snapshots can be trimmed (--inline-snapshot=trim)

From the docs:

trim removes every snapshots form the storage which is not referenced with external(...) in the code.

However if I run pytest with --inline-snapshot=trim it doesn't do anything. I have 11 external files indeed, so I suspect inline-snapshot doesn't detect external calls in my code:

snapshots = snapshot(
    {
        (
            ("annotations_path", "brief"),
            ("separate_signature", True),
            ("show_signature", False),
            ("show_signature_annotations", False),
            ("signature_crossrefs", False),
            ("unwrap_annotated", True),
        ): external("3603a08e7614*.html"),
        ...
    }
)
pawamoy commented 2 months ago

Can't reproduce while debugging. external calls are correctly identified. Maybe an issue with pytest-xdist again. UPDATE: yep, no such message when disabling pytest-xdist.

pawamoy commented 2 months ago

Quick thought: if inline-snapshot finds uses of external by searching in the files collected by the current pytest process, then it's possible that one of the process didn't run a single test from my module using external, and therefore inline-snapshot finds no reference to the existing snapshots. A solution would be to somehow check every test file used across the run (multiple process through xdist). Or maybe even all tests file, period, because if I unselect tests using external, that doesn't mean my snapshots are unused.

15r10nk commented 1 month ago

0.9.0 contains now a check if you use xdist at the same time.

I think there are several issues with the current implementation of outsource. I have some ideas how the DX could be improved and wrote it down (#86). Please tell me if you have some feedback.