SublimeText / UnitTesting

Testing Sublime Text Packages
MIT License
112 stars 32 forks source link

Add support for `yield self.assert*(...)` #273

Closed giampaolo closed 5 months ago

giampaolo commented 5 months ago

I'm not sure how feasible this is to implement, but here goes. When writing unit tests for ST it's very common to wait for a condition to happen. E.g.:

    def test_something(self):
        self.window.run_command("show_panel", {"panel": "console"})
        yield lambda: self.window.active_panel() == "console"

Despite this works, it does not give any hint on what active_panel() returns if the condition fails due to a timeout. It would therefore be nice to do this instead:

    def test_something(self):
        self.window.run_command("show_panel", {"panel": "console"})
        yield lambda: self.assertEqual(self.window.active_panel(), "console")
deathaxe commented 5 months ago

Waiting for a condition to continue test case execution (snippet 1) is fundamentally different from running an assertion (snippet 2).

A condition is something which must be met, but may take a while to do so due to asynchonously tasks needing to finish in background. It's main purpose is synchonization. The callable passed is polled until returning true. As self.assert... methods don't return anything and would raise exceptions if condition is not meat they are not applicable in this context.

Conditions can be passed as dictionary. Maybe passing an optional error message can be implemented, which is displayed if timeout appears.

    def test_something(self):
        self.window.run_command("show_panel", {"panel": "console"})
        yield { 
            "condition": lambda: self.window.active_panel() == "console",
            "period": 100,
            "timeout": 5000,
            "timeout_message": "Console not activated"
        }
giampaolo commented 5 months ago

The callable passed is polled until returning true. As self.assert... methods don't return anything and would raise exceptions if condition is not meat they are not applicable in this context.

My proposal was to keep catching AssertionError as a synchronization mechanism as in:

def call_until(assert_call, timeout=5):
    stop_at = time.monotonic() + timeout
    while True:
        try:
            return assert_call()
        except AssertionError:
            if time.monotonic() > stop_at:
                raise TimeoutError
            time.sleep(0.01)
deathaxe commented 5 months ago

Interesting idea. When assuming assertions to always return None and normal conditions only True or False, it should be doable with hopefully low risk on breaking anyones test cases.

Was too focused on "missing a message" and the aspect of conditions vs. assertions, before.