Open jorenham opened 1 month ago
Thanks, @jorenham! As one might surmise, updates from the various metropoli of performant runtime type checking are slow to reach these backwaters of Appalachia, so we very much appreciate it when kind travelers bring news as they pass through. And please do not worry yourself on the matter of provoking offense. I try to reserve that for much more serious matters which largely but tragically not absolutely exist outside of software development.
It is deliberate that numerary
describes itself as a "codified rant". The fundamental problem is that official "development" and I use that term liberally as applied here at the intersection of operations and types is a hot mess beyond the most trivial cases and sometimes even then. Every time I've tried to make use of a structure or feature in the standard library whose documentation is simple, authoritative, and clear, I find that the reality is anything but. There is always some unfinished corner or undeclared edge case that frustrates fitness for the purpose explicitly envisioned by the feature's designer(s).
No matter the path down which I tread, I find myself in the middle a murky swamp impassible with flora, fauna, and developers, who declare my very presence as an intrusion often without patience or explanation, as if under the assumption that non-residents of that bog should magically just know to temper our understanding of the official documentation and various accepted PEPs through the lens of whatever in-the-moment, jargon-laden sentiment or "reasoning" again, used liberally casually littered throughout developer forums, mailing lists, or buried in some comment in some bug report that was closed as "not planned" five years ago. Through lands far and wide, I have tracked and pursued ABCs, various Supports*
conventions, the numeric type hierarchy, generics (to the extent Python has them), protocols, operations and dunder methods, and other head spinning chicanery like that which you've discovered. At the end of every exploration, I encounter the same small group of wizards declaring that I "shall not pass" or that my way lies "back the way I came" and that I should either be satisfied to limit my use to int
s and float
s or commit to explicit support of a particular third party mathematics library most of which are thankfully at least more forthcoming about what is and isn't supported.
I have a fantasy that Python 4 will be the typing-finally-works-coherently-and-we-should-be-clear-what-is-deprecated-by-getting-rid-of-everything-Python-developers-no-longer-wish-to-maintain release, but that isn't based on anything in reality and I'm not holding my breath.
In other words, I wonder if your prompt should really read, "Is ~the protocol caching~ numerary
still needed?"
optype
just might be the new numerary
!)I would love to see performance numbers comparing Beartype's CachingProtocolMeta
to 3.12's improved library version, but it's possible that numerary
is no longer needed (or perhaps, more accurately, a dead-end). What I was essentially trying to capture is a way for things to declare capabilities and then allow for other things to offer functionality around those capabilities.
Unless I've misunderstood, this is precisely what optype
does!
I still have yet to figure out is how to constrain secondary functionality based on the capabilities of the things on which that functionality should operate. That's probably too vague to be coherent, so an example may be in order. Consider:
R = TypeVar("R")
T = TypeVar("T")
type CanTMulR[T, R] = CanMul[T, R] | CanRMul[R, T]
type CanTAddR[T, R] = CanAdd[T, R] | CanRAdd[R, T]
class MyThingThatDoesMathLikeThingsWithOtherThings(Generic[T]):
def __init__(self, thing: T):
self._thing = thing
def do_the_multiplication_thing(self, other: CanTMulR[T, R]) -> ???:
if isinstance(other, CanRMul):
return self._thing * other
else:
return other * self._thing
def do_the_addition_thing(self, other: CanTAddR[T, R]) -> ???:
if isinstance(other, CanRAdd):
return self._thing + other
else:
return other + self._thing
There are two specific problems I don't yet know how to solve. (To be fair, numerary
doesn't solve these either.)
MyThingThatDoesMathLikeThingsWithOtherThings.do_the_multiplication_thing
is unavailable if CanTMulR[T, R]
doesn't resolve, but other methods may still work.MyThingThatDoesMathLikeThingsWithOtherThings.do_the_multiplication_thing
based on what T
and R
are (i.e., reveal_type(MyThingThatDoesMathLikeThingsWithOtherThings(2).do_the_multiplication_thing("I")
yields str
). Maybe something like ResultTypeForMul[T, R] | ResultTypeForRMul[R, T]
? Although that isn't quite right. What I really mean is ResultTypeForMulOrRMulIfMulNotAvailable[T, R]
.I don't know whether either problem is solvable given the current state of the world? The second may require some kind of "registration" mechanism? As an aside, division (__div__
and __rdiv__
) has always been my go-to case, because it violates the naive assumption that A __op__ B
always yields one of A
or B
(e.g., int / int -> float
).
Complicating a simple reversion to the standard library version is that (much of) CachingProtocolMeta
is actually part of beartype
. It's true that numerary
provides its own work-arounds for declaring/overriding inclusion/exclusion for runtime checking, but those rely heavily on the proprietary internals of Beartype's implementation. There are a couple options I can see, pending some performance validation:
beartype
's _CachingProtocolMeta
and pin numerary
's reliance on beartype
to an upper bound where it was still present (and possibly declare numerary
as officially deprecated and use its README
to advocate for adopting optype
instead :partying_face:).beartype
's _CachingProtocolMeta
and remove numerary
's include
/exclude
mechanism altogether (which means runtime type checking would be ... erm ... less correct :grimacing:).beartype
's _CachingProtocolMeta
and re-implement numerary
's include
/exclude
mechanism such that it is self-contained within numerary
.Assuming there's no longer any performance benefit to beartype
's _CachingProtocolMeta
, the last option feels like the "right" thing to do assuming anyone is actually using numerary
for anything useful, which is probably a stretch.
Wow, that must be the best response I've ever gotten; you have my thanks π .
Every time I've tried to make use of a structure or feature in the standard library whose documentation is simple, authoritative, and clear, I find that the reality is anything but.
This is my experience as well: Seemingly simple things like binary operators, or iterations, are ridiculously to correctly type, often resulting in a mess of type aliases, protocols, overloads, typeshed PR's that get rejected for fallacious reasons, and angry neighbors telling you to stop screaming at your screen at 4 a.m...
I encounter the same small group of wizards declaring that I "shall not pass" or that my way lies "back the way I came"
I think that I have encountered this buggy endboss a couple of times as well. It's one of their special attacks that gets me, usually its either "dogma beam" or "fallacy cannon", or when they're at 1HP, they exploit a glitch that replaces them with an another, almost identical, boss.
In other words, I wonder if your prompt should really read, "Is ~the protocol caching~
numerary
still needed?"
Yes, it is!
The numbers
stdlib creates more problems than it solves, collections.abc
doesn't seem to care about numbers, and there's that thing where at runtime you have that bool <: int
, but whilst type checking it's treated as if complex <: float <: int <: bool
(type-checkers allow it if you e.g. pass a bool
to some def f(x: int)
, which is just rude to those with a bool
-allergy).
Plus, there are more that one type of number; e.g. fractions.Fraction
, decimal.Decimal
, or any of the subtypes of numpy.number[Any]
.
And even when you dump you integer-like input into an int(x)
, you can't just accept everything that implements __int__
or __index__
; e.g. numpy.ndarray
implements both, but unless it is 0-dimensional, it'll raise a TypeError
; which is impossible to type from within numpy
itself (or at least until numpy fully supports shape typing).
I'm not sure whether this can all be covered by numerary
, but I know that optype
is too low-level do even come close.
Relying on typeshed alone won't do you much good either, as they're just typed versions of the runtime.
But what's needed is a bridge between the world of runtime and the world of typing.
That bridge will have to be shaped like a bunch of typing protocols that are flexible, but not too loose: Stiff bridges break, loose bridges wobble.
And currently, many people attempt to use the numbers
lib as a bridge, even though it's made of moldy cardboard and duct-tape.
There are two specific problems I don't yet know how to solve. (To be fair,
numerary
doesn't solve these either.)
- How to declare/detect at least statically, but ideally at runtime as well that
MyThingThatDoesMathLikeThingsWithOtherThings.do_the_multiplication_thing
is unavailable ifCanTMulR[T, R]
doesn't resolve, but other methods may still work.- How to infer the return type of (e.g.)
MyThingThatDoesMathLikeThingsWithOtherThings.do_the_multiplication_thing
based on whatT
andR
are (i.e.,reveal_type(MyThingThatDoesMathLikeThingsWithOtherThings(2).do_the_multiplication_thing("I")
yieldsstr
). Maybe something likeResultTypeForMul[T, R] | ResultTypeForRMul[R, T]
? Although that isn't quite right. What I really mean isResultTypeForMulOrRMulIfMulNotAvailable[T, R]
.
How about something like this?
from __future__ import annotations
from typing import Any, Generic, TypeVar, overload
from optype import CanMul, CanRMul
_T_co = TypeVar('_T_co', covariant=True)
_R = TypeVar('_R')
_RHS = TypeVar('_RHS')
class SomeThingThatDoesMathLikeThingsWithOtherThings(Generic[_T_co]):
def __init__(self, thing: _T_co) -> None:
self._thing = thing
@overload
def do_the_multiplication_thing(
self,
other: CanRMul[_T_co, _R],
) -> _R: ...
@overload
def do_the_multiplication_thing(
self,
other: CanMul[_T_co, _R],
) -> _R: ...
@overload
def do_the_multiplication_thing(
self: SomeThingThatDoesMathLikeThingsWithOtherThings[CanMul[_RHS, _R]],
other: _RHS,
) -> _R: ...
def do_the_multiplication_thing(
self,
other: CanRMul[_T_co, _R] | CanMul[_T_co, _R] | Any,
) -> _R:
if isinstance(other, CanRMul):
return self._thing * other
if isinstance(other, CanMul):
return other * self._thing
return self._thing * other
Complicating a simple reversion to the standard library version is that (much of)
CachingProtocolMeta
is actually part ofbeartype
. It's true thatnumerary
provides its own work-arounds for declaring/overriding inclusion/exclusion for runtime checking, but those rely heavily on the proprietary internals of Beartype's implementation. There are a couple options I can see, pending some performance validation:
Purely in terms of isinstance
speed, the Python 3.12+ runtime-protocols are great.
But because of that, they are very imprecise, and often result in false-positives, of which many should be easily avoidable.
So consider the following standard boilerplate code for some regular business application:
from typing import SupportsIndex, Self
class SomethingPositiveYouCanCountOn:
"""
>>> _ = SomethingPositiveYouCanCountOn()
>>> +_
+_
>>> ++++_
++++_
"""
def __init__(self, /, *, _start: SupportsIndex = 0) -> None:
self._value = int(_start)
def __pos__(self) -> Self:
return type(self)(_start=self._value + 1)
def __repr__(self) -> str:
return '+' * self._value + '_'
class SomethingThatLooksPositiveButIsNot:
__pos__ = NotImplemented
How can we want to predict whether some x
can have a +
in front of it, but without actually trying it (i.e. what you do in practice if you want to avoid side-effects, e.g. sloths taking over the world, forcing humans to act as their trees).
Runtime-protocols are the way to go here:
>>> from optype import CanPos
>>> isinstance(SomethingPositiveYouCanCountOn(), CanPos)
True
Now let's check SomethingThatLooksPositiveButIsNot
:
>>> isinstance(SomethingThatLooksPositiveButIsNot(), CanPos)
True
Ouch.
So let's check what every 3-year old python developer knows: that even if __pos__
is implemented, +x
can only be used on the instance, but not on the type itself
>>> isinstance(SomethingThatLooksPositiveButIsNot, CanPos)
True
So yes, a better typing.Protocol
might be a good idea π¦₯
So this was kind of rad (cc @leycec; hat tip to @AlexWaygood for the beartype
mention; apologies that I'm only just now getting caught up on all of this):
Given that behaviour, it could well be reasonable to slap a cache on the whole
_ProtocolMeta.__instancecheck__
call, similar to the waybeartype
does it... .
I also missed this:
FYI, all the changes that have been made to
typing_extensions.Protocol
in v4.6.0 are also going to be included intyping.Protocol
in Python 3.12. The changes (to bothtyping.Protocol
andtyping_extensions.Protocol
) were mostly made in order to achieve a 20x speedup forisinstance()
checks against runtime-checkable protocols.As well as the fact that
typing_extensions.Protocol
is much faster thantyping.Protocol
on <3.12,typing_extensions
also backports several significant bugfixes that have been made totyping.Protocol
in recent Python versions, but haven't been backported as far as Python 3.8.
If I'm reading that right, using typing_extensions.Protocol
for Python <3.12 may very well be good enough to sunset beartype.typing._typingpep544._CachingProtocolMeta
. I still want to do some perf testing, though. This would have to be optional, meaning maybe beartype
could use the typing_extensions.Protocol
if available, but I don't think it would (or should) tolerate a dependency.
However, based on your examples, @jorenham, (which are unsurprising to those who know how __instancecheck__
is implemented), I think numerary
's registration bit may still be needed. I would love it if this could be done via Python's existing machinery. However, while ABCMeta
allows explicit registration to avoid false negatives, it does not afford explicit exclusion to avoid false positives like the ones you identify. Without realizing it, numerary
ended up trying to implement what turned out to be an old idea (cf. this, this, and this).
I'll try to tinker some more once I can carve out some time.
Yeah, would definitely encourage y'all to do detailed perf testing before making any decisions. We made a ton of optimisations to isinstance()
checks against typing.Protocol
in 3.12 and they're way faster than they were in 3.11, but:
isinstance()
checks against other classesbeartype
because we have to be backwards compatible with behaviour on previous Python versionstyping_extensions
: some of the optimisations were achieved by optimising inspect.getattr_static
, and those couldn't be realistically backported to typing_extensions
.There's a benchmark script in the PR description for https://github.com/python/cpython/pull/103280#issue-1655797096 that you might find useful
Since Lithuanian breakdancer Dominika BaneviΔ just got robbed in the B-Girls Olympics gold medal Breaking battle, I must now console myself with some fast-acting GitHubbing. I drain my sorrows in keyboard warrioring. </ahem>
What I meant to say was:
Thanks so much, @AlexWaygood! That PR benchmark script is especially awesome. If CPython's official typing.Protocol
implementation is anywhere near @beartype's mostly unmaintainable beartype.typing.Protocol
implementation, I'll delightedly kick the latter to the curb. While that cat is fast as lightning, @beartype's current approach is also:
beartype.typing.Protocol
silently ignores class methods, whereas typing.Protocol
appears to properly detect class methods. In theory, beartype.typing.Protocol
could be fixed to properly detect class methods as well. In practice, it might be preferable to just amputate this gangrenous limb from the @beartype codebase and let somebody else who is smarter than me maintain this special madness.Will @leycec actually do something? Will @posita save the bitter day? Stay tuned as we all just watch more caustic Olympics.
Since Python 3.12 runtime-checkable
typing.Protocol
's use caching and some other tricks to achieve a ~20x speedup: https://github.com/python/cpython/issues/102433For older Python versions,
typing-extensions
provides aProtocol
variant that includes (most of) these speedups (docs). For what it's worth: It is used in theoptype
library, which I use for several of my (other) open- and closed-source projects, but I haven't had any problems with it at all (even though I tend to break everything typing related (not a threat BTW)) π€·π».I haven't done a benchmark, but regardless of the outcome, this could perhaps be a nice opportunity to make
numerary
more focused towards what all the cool kids are talking about these days: typing in Python π. Even fusion-powered quantum-blockchain AI has got nothing on us!~
Hoping this banal participial closing causes no offense, I am,
Typically,
Joren S. Hammudoglu, Typer of Things