initialcommit-com / git-sim

Visually simulate Git operations in your own repos with a single terminal command.
GNU General Public License v2.0
4.19k stars 109 forks source link

Use parametrization to simplify tests. #100

Closed ehmatthes closed 1 year ago

ehmatthes commented 1 year ago

Working on #99, I played around with parametrization. I didn't think it was going to help, but it is pretty compelling. Instead of 18 separate test functions, there's only one test function. All of the commands that are being tested are moved to one list, git_sim_commands:

git_sim_commands = [
    # Simple commands.
    "git-sim add",
    "git-sim log",
    ...

    # Complex commands.
    "git-sim branch new_branch",
    "git-sim checkout branch2",
    ...
]

Then test_command() is called, with each of these passed as an argument called raw_cmd:

@pytest.mark.parametrize("raw_cmd", git_sim_commands)
def test_command(tmp_repo, raw_cmd):
    """Test a  git-sim command.

    This function works for any command of the forms
      `git-sim <command`
      `git-sim <command> <arg>`
    """
   ...

Pros:

Cons:

When a test fails, you still get really clear information about what specific behavior is broken:

============ short test summary info ==========================================================
FAILED tests/e2e_tests/test_core_commands.py::test_complex_command[git-sim cherry-pick branch2]
- AssertionError: assert 'git-sim-cherry-pick' in
  '/private/var/folders/gk/y2n2jsfj23g864pdr38rv4ch0000gn/T/pytest...
============ 1 failed, 4 passed, 18 deselected in 28.28s ======================================

Overall, I think this is a change worth making. But that's entirely up to you, and I'd be quite happy to keep helping out with or without this change.

initialcommit-io commented 1 year ago

Love it. When I initially saw the test suite it did seem a bit redundant to have a separate method for each command, considering they are so similar.

If at some point we end up needing various git-dummy repo layouts to support the simulation of various git subcommands, we may need to find a way to parameterize the git-dummy options/flags as well, but that can put that on the backburner until that situation crops up.

Also (semi-unrelated) but I hear your points about keeping this first version of the test suite as simple as possible. But for the commands like add, rm, mv, stash etc the current test simulations don't fully reflect the functionality since no files exist to be added, removed, stashed, etc. Well actually they do exist for stuff like rm and mv since we can just execute those commands on existing dummy files, but for git-sim add and stash I am planning to update git-dummy to allow modifying a local file and/or creating a new untracked file which can be used for the test simulations.