odlgroup / odl

Operator Discretization Library https://odlgroup.github.io/odl/
Mozilla Public License 2.0
375 stars 105 forks source link

How / to what extent should we use type hinting? #1654

Open leftaroundabout opened 3 months ago

leftaroundabout commented 3 months ago

Python is now in some sense more of a statically-typed programming language than it was at the time when ODL was first developed. While Python types are still in principle a runtime concern, the language allows and encourages adding type hints to functions. Even though these are neither necessary for compilation or performance nor provide the powerful guarantees that types give in static languages, I find that type hints often make it easier to understand how a function/method is supposed to be used. They are more immediate to see than Sphinx docs, and avoid the ambiguity of argument-documentation phrased in natural language.

Unfortunately, many Python functions exploit dynamic typing (in particular, duck typing and ad-hoc isinstance checking) in a way that makes it hard or impossible to decide what "the type signature" of a function should be, and ODL in particular makes widely use of this possibility. This makes addition of type hints a nontrivial endeavour.

On the other hand, ODL is also a library that could particularly benefit from type hints: I personally think of the whole infrastruction of spaces and elements as a way to encode mathematical information into types of the programming language. This could be made more explicit if those types actually appeared in the signatures of functions / operators.

I see a couple of ways to move from here:

  1. Leave existing ODL functions as they are. Only add type hinting in new features and only if there is an obvious way. This is obviously the least-effort version and it would avoid ending up with incorrect signatures. Of course we also would not get the advantages though, and ODL is and would remain on many sides hard to understand, perhaps harder than it needs to be.
  2. Add type hints incrementally, to functions where it has been deemed necessary and there is a reasonably clear understanding what type it should be.
  3. Schedule an effort to make most of ODL type-hinted. This would mean going through every source file, determining as best as possible the type signature of all the functions/methods, and writing them out as type hints.

For both 2. and 3. there would be in some cases judgments to be made – a tradeoff in precision vs. practicality.

  1. If every possible way how a function can be called is to be considered, then this would often correspond to very complicated types like Union[bool, Iterable[Union[bool, Tuple[bool,bool]]]]. Such a type can still be useful, but especially to a beginner it is probably harder to understand than just looking at some examples for how the function can be used. In this case the type might overall make the user experience worse.
  2. Encode only the most common, most conceptually easy, and/or most general use case in the type hint and then list in the Sphinx docs that there are alternative ways of calling the function.
    This might be a good compromise to get only the advantages of type hints without unnecessary bloat/confusion. However it would also mean the types are lying somewhat about how the function can be used, and it could cause static analysers to flag use cases that are actually handled just fine by the function.
  3. Use precise types, but keep them simple by other means.
    This is an option worth considering not only in the context of type hints, also generally: instead of ad-hoc "you can call this with a scalar or tuple of bla bla", one could make a dedicated class for this argument.
    This can work well, in particular with the advances Python also has made to algebraic data types. The arguments could become better to understand and the type hints clear and concise, but on the flip side this would mean an actual breaking change in calling convention, quite a lot of boilerplate in ODL, and typically also more verbose function invocations.