Enforce types during tests

[timobrembeck]

Motivation

At the moment, types are only checked statically via mypy. mypy does not execute the code, but just analyze it. It would be cool to check the types during the tests and make sure that all variables really have the expected types during runtime.

Proposed Solution

Not sure how we could achieve this, one possibility is the library strongtyping which provides a decorator @match_typing which throws an exception if a function call does not match its type annotations.

However, adding this to every function in the entire code base seems like a lot of overhead. Maybe we could achieve this by patching this dynamically just before the tests are run? (Either via monkeypatching or via a stupid regex-replacement of def occurrences...)

Alternatives

Rely on the current mypy checking

[david-venhoff]

This might also be useful, even though it does not exactly what we want: https://mypy.readthedocs.io/en/stable/stubtest.html

[timobrembeck]

This might also be useful, even though it does not exactly what we want: https://mypy.readthedocs.io/en/stable/stubtest.html

Good point! This is also what I used for the initial baseline of the types, but it required a lot of manual adjusting, would probably complicated to integrate into an automated workflow. But yes, we should investigate this when tackling this issue.