feature: API checks: Check for types compatibility

[pawamoy]

Following the discussion in #75, we released a first version of the API breakage detection feature, which does not check if types (parameters types, return types) are compatible between the old and new code. This issue is here to discuss about adding support for checking types compatibility.

It's already possible at runtime with beartype, so maybe we could use in the inspector. But for the visitor we need something static.

[pawamoy]

Moving this here from the README:

• List of things to consider for breaking changes
  • [ ] Changed type of parameter
  • [ ] Changed type of public module attribute
  • [ ] Changed return type of a public function/method
  • [ ] All of the previous even when parent is private (could be publicly imported or assigned somewhere), and later be smarter: public assign/import makes private things public!
  • [ ] Inheritance: removed, added or changed base that changes MRO

[pawamoy]

I wonder: could we somehow recurse into objects to check that they implement the same "protocols"? Example:

class Foo:
    def method1(self): ...
    def method2(self, param1: int, param2: int): ...

class Bar:
    def method2(self, param1: int, param2: int): ...

class Baz:
    def method1(self): ...
    def method2(self, param1: int): ...

class Qux:
    def method1(self): ...
    def method2(self, param1: str, param2: int): ...

class Quux:
    def method1(self): ...
    def method2(self, param1: int, param2: int): ...
    def method3(self): ...

def f_old(param: Foo): ...

def f_new(param: Bar): ...  # breaks: removed method1
def f_new(param: Baz): ...  # breaks: removed param2 on method2
def f_new(param: Qux): ...  # breaks: changed type of param1 on method2 (int -> str is a "known" breakage)
def f_new(param: Quux): ...  # doesn't break: added method3

I imagine a type_compatible(self, other) method on objects that uses next(find_breaking_changes(self, other)) and returns False if a breakage is yielded, True if StopIteration is raised.

[pawamoy]

Or maybe there's a way to hook into mypy...

[pawamoy]

We would need to match protocols in all expression parts.

In dict[MyStr, This] vs. MyDict[str, That]:

• dict and MyDict must match
• MyStr and str must match
• This and That must match

To compare objects, we'd have to dynamically load external objects (but only if they weren't already used). Examples:

• param: list[otherpkg.Thing] vs. param: Sequence[otherpkg.Thing]: here we see that otherpkg.Thing is the same object (canonical path) before and after, so we don't need to load it
• param: otherpkg.Thing vs. param: set[str]: here we'd need to load otherpkg.Thing
• param: set[str] vs. param: otherpkg.Thing: here we'd need to load otherpkg.Thing again

Or we could expect users to pre-load these modules and soft-fail otherwise.

By the way, how to compare set[str] to Thing? Should Thing inherit from a generic with str type? Or should all the equivalent parameters be typed str, and if they're typed T (type alias? how to check that?) then only check generics?