pschanely / hypothesis-crosshair

Level-up your Hypothesis tests with CrossHair
MIT License
8 stars 0 forks source link

Issue with multiply registered contracts #1

Closed pschanely closed 7 months ago

pschanely commented 8 months ago
import time as tm, pytest
from functools import wraps
from hypothesis import given, settings, strategies as st

@settings(backend="crosshair")
@given(st.text("abcdefg"))
def test_hits_internal_assert(x):
    assert set(x).issubset(set("abcdefg"))

@pytest.fixture(scope="function")
def _patch_time(monkeypatch):
    def time():
        nonlocal current_time
        current_time += 0.1
        return current_time

    current_time = tm.time()
    monkeypatch.setattr(tm, "time", wraps(tm.time)(time))
    monkeypatch.setattr(tm, "monotonic", wraps(tm.monotonic)(time))
    # monkeypatch.setattr(tm, "perf_counter", wraps(tm.perf_counter)(time))

@settings(backend="crosshair")
@given(n=st.integers())
def test_crosshair_does_not_like_monkeypatching(_patch_time, n): ...
    # crosshair.register_contract.ContractRegistrationError: 
    # Pre- and postconditons and skip_body should not differ when registering multiple contracts for the same function

Originally posted by @Zac-HD in https://github.com/HypothesisWorks/hypothesis/issues/3914#issuecomment-1987097328

pschanely commented 8 months ago

Fascinating. (cc @Zac-HD for amusement) Because CrossHair applies different contracts to these functions (time can produce any float, monotonic can only produce increasing values), it balks when time.time is time.monotonic (CrossHair intercepts calls via function identity)

Either way, we don't want this kind of advanced behavior when running under hypothesis. (in normal CrossHair, the example will be explained with a bit of code to artificially patch the time functions to reproduce the necessary values, but under hypothesis, I imagine the inputs will just fail to reproduce) I'm sure the random module will have similar issues. I'll investigate the best way to turn these off for the plugin!

pschanely commented 7 months ago

Fixed in v0.0.2!