search

koaning / uvtrick

A fun party trick to run Python code from another venv into this one.
MIT License
153 stars 6 forks source link

Snapshot tests of examples #19

Closed lmmx closed 2 months ago

lmmx commented 2 months ago

This PR adds snapshot tests that "pin" the exact output of tests, and store it in the test files automatically (broken out of previous PR #9 as requested)

This seemed like a perfect candidate for inline-snapshot, providing both confidence in the results staying the same, and the ability to inspect the outputs. The inline-snapshot plugin helps easily create/review/fix 'snapshotted' values

The snapshots come from an if "pytest" in sys.modules: block, which prints the script metadata and the contents of the intermediate script.

I used a utility function in conftest.py, fmt_output(), to postprocess this 'log', which I then assigned to the pytest namespace (since you can't use a pytest fixture for this). This could live in the library module but it's not used at runtime so felt unnecessary.

The 'preamble' (file name and function name) is stripped of newlines so that it doesn't run over onto a newline and is therefore visually distinct from the code block (which stays unindented thanks to the backticks either side, and is therefore possible to copy and paste into an interpreter)

koaning commented 2 months ago

I have never really used snapshots before, and while I can see them be beneficial in some usecases... I am kind of wondering if it is really needed here? It is very early in this project so I am a bit hesitant to introduce tools unless they are clearly needed.

lmmx commented 2 months ago

Fair enough, it was helpful to ensure I didn't go off track, I can keep snapshots separately with a uv workspace or something in that case :thinking: Something to think about!