Closed DylanLukes closed 1 month ago
As for the original motivating case... which is admittedly a pretty odd thing to do, the goal was to define a well-typed Singleton
metaclass/base-class which supported a @singletonmethod
decorator on methods. This would behave much like @classmethod
except the resulting method would always be called on the singleton instance, regardless of whether it's called on the class or an instance.
But the wrinkle came in making this work for Singleton
s that inherit from non-Singleton
abstract base classes...
There are probably all kinds of reasons this should not be done, and I wouldn't be surprised if the typechecker shouldn't be allowing this...
The (frankly somewhat evil code) looks something like this:
My kingdom to avoid a # pyright: ignore[reportIncompatibleMethodOverride]
! 😅
You're wandering into very murky areas of the typing spec here.
A few observations:
BaseB
, you're declaring foo
as an instance variable. I suspect that you intended to declare it as a ClassVar
? A def
statement (which you use to define foo
in DerivedA
) describes a class variable, not an instance variable. So if you want these two cases to be even close to equivalent, you'd need to at least mark BaseB.foo
as a ClassVar
.Callable
variable in a Protocol or a non-protocol class (whether it's defined as a ClassVar
or not) should be treated. These are tricky because they are treated differently depending on whether they are bound to a class or a class instance (an object). This is another thing that I'd like to see specified at some point in the future.DerivedA.foo
or DerivedB2.foo
, but it does generate one for DerivedB1.foo
. As you've noted, pyright generates an error for both DerivedA.foo
and DerivedB1.foo
but does not generate one for DerivedB2.foo
. Not surprisingly, the two type checkers differ in how they behave here. That's not surprising given the lack of specifications.In general, if you start to intermix Callable
variables and def
statements, you're going to see some oddities. I recommend avoiding this if possible. This gets especially hairy if you throw ParamSpec
-based decorators into the mix.
Until and until some of the above gray areas are clarified in the typing spec, I'm reluctant to change the logic in pyright in these areas. Doing so is likely to just create churn for existing pyright users, and it would likely need to change again once when the typing spec is updated.
I'm therefore going to close this issue for the time being. I'm willing to revisit it if and when the above rules are clarified in the typing spec. If you would like to participate in that spec'ing process, the Python typing forum is a good place to start. This page provides information about the process we use to consider changes and additions to the typing spec.
Thank you for the detailed reply. Confirms my suspicions but also offers some food for thought.
I'm reluctant to change the logic in pyright in these areas.
Yeah, absolutely. I'm clearly wandering off the well-charted areas of the map here so locking anything in seems unwise.
The typing spec also doesn't currently define most of the rules regarding instance methods, class methods, and static methods and how they should be interpreted when passed through a ParamSpec. This is another thing I'd like to see specified.
I wonder if it might (eventually) be possible to formalize a set of descriptor protocols and then if not make absolute decisions at least formalize which component specifications are included. That is, to say "this is the protocol for a descriptor representing a class/instance variable/method" and then be able to specify the behavior of the type checker in terms of those primitives. For example "absent explicit class var annotation, a field of type T
is implicitly "promoted" to Union[ClassVar[T] | InstanceVar[T]]
where both types have overlapping subsets of overloads on __get__
/__set__
/__delete__
or something like that.
Maybe I'm off base here but it seems like the kind of behavior I was trying to achieve would be more achievable if such descriptor protocols could be explicitly stated in Protocol
s, and it might also lead to some clarity on issues such as "when can a field implement a protocol property or vice versa", since while the two look the same in many use cases they differ in the behavior of the underlying descriptors. For example, if I could in a type safe manner specify "I really just want a thing that I can __get__
with this type when access on an instance, that type when accessed on the class, and some other type when accessed independently through __dict__
or the C API. [1]
I might also just be a bit out of my depth in the above. But, it seems to me that there ought to be some sort of formalization that works, though it's not clear to me if the intersection of the semantics of that formalization and subtyping in Python are easily reconcilable (my guess is they're not).
The discussion about PEP 746 and __supports_type__
seems potentially relevant as it pertains to Callable
types (or even MethodType
and its ilk).
[1] One obstacle I've run into here is that it's not possible to use Concatenate
only with P.args
for a P
ParamSpec
, so one can't write an overload for a method which might take a self
parameter (Concatenate[T, P.args]
) or not P.args
. If this were possible, it might be feasible to type such callable descriptors as above to have different __call__
signatures depending on their usages.
I think these ideas are worth exploring. However, the Python typing forum would be a better place to have this discussion. This will give it the attention and visibility of the full Python typing community.
Overview
This is perhaps more than a little bit esoteric, as it involves the intersection of
ABC
/Protocol
,Callable
, and descriptors. The original motivating case is a bit more complicated and I'll include it in a follow-up comment to explain how I even got to finding this odd edge case.In short, it is not possible (and for, I think, good reason) possible to implement an
@abstractmethod
with a descriptor, even if that descriptor isCallable
. However, it seems like it should be possible to implement aCallable
field on a Protocol using aCallable
matching descriptor. It turns out this is possible, but only if one avoids using@decorated
method syntax.In spite of
reveal_type
disclosing exactly the same typing, Pyright behaves differently depending on the syntax used.Example
First, let's define a sneaky class-based decorator:
In practice this has no immediately effect when applied to a method, except that the method can no longer be called through a
__dict__
lookup and passing the instance in as the first argument. So, it's very close to behaving identically to the original method, but it is not a method.Inheritance: Case A
It being the case that this callable descriptor is not actually a method, and this would fail at runtime, it makes sense that attempting to implement a
Protocol
orABC
) method with aboundmethod
descriptor should fail. The signatures (even taken just as callables) don't match (differing positional-only/bound parameters).So far so good. This is the correct judgement on behalf of the type checker.
Inheritance: Case B
Now, what if the base protocol does not define a method but instead a
Callable
field, which ourboundmethod
descriptor does properly implement? Well, oddly enough in this case it's syntax-dependent, even though the resulting classes are effectively identical.Let's try it two ways, as a decorated method (
DerivedB1
) and using an extra private method and initializing the descriptor directly (DerivedB2
).And now here we get the following type checking output.
Notably, we do not get a type error for
DerivedB2
, despite the fact that the types offoo
as revealed inside the class definition and outside in usage are indistinguishably isomorphic betweenDerivedB1
andDerivedB2
.This seems a bit odd to me, it seems like intuitively, it shouldn't matter if decorator syntax is used, or if the descriptor is initialized directly in the class body. That is,
@decorator; def foo(): ...
anddef _foo(): ...; foo = decorator(_foo)
should be typed equivalently, no?System Details