pschanely / CrossHair

An analysis tool for Python that blurs the line between testing and type systems.
Other
1.04k stars 49 forks source link

Higher-order function is not covered by `crosshair cover` #174

Closed pschanely closed 2 years ago

pschanely commented 2 years ago

Expected vs actual behavior The TIM Improves Merging project found that crosshair could not generate two inputs to cover each branch of the following function (see additional discussion in the report PDF). Here, we see only the first branch is covered:

$ cat callableex.py
from typing import Any, Dict, List, Optional, Tuple, Generator, Callable, Iterable, IO, TypeVar, Mapping

def lambdaFn(a: Callable[[int], int]):
    if a:
        return a(2) + 4
    else:
        return "hello"

$ crosshair cover callableex.lambdaFn
lambdaFn(lambda *a: None)

One might expect to see inputs that cover both paths in the if statement.

pschanely commented 2 years ago

(I'm self-reporting and self-answering here, for the benefit of future CrossHair users)

In order to generate None CrossHair demands that the type have the Optional[] wrapper. This might be a common sort of stumbling block - mypy only recently made --no-implicit-optional the default behavior.

Adding Optional gives us the two paths we expect:

$ cat callableex.py
from typing import Any, Dict, List, Optional, Tuple, Generator, Callable, Iterable, IO, TypeVar, Mapping

def lambdaFn(a: Optional[Callable[[int], int]]):
    if a:
        return a(2) + 4
    else:
        return "hello"
$ crosshair cover callableex.lambdaFn
lambdaFn(lambda *a: None)
lambdaFn(None)