Deriving builtins from ABCs implies incorrect and problematic metaclass structure

[elazarg]

typeshed claims that str derives from Sequence and therefore indirectly from collections_abc.Sized whose metaclass is declared to be ABCMeta. This is done as a type-checkable alternative to Sequence.register(str) which handles isinstance and issubclass calls.

However derivation and register are not indistinguishable, since the metaclass structure is different. issubclass(type(str), ABCMeta) should be False, otherwise the definition class A(str, Enum): pass would fail at runtime due to inconsistent metaclass structure:

>>> from collections import Sized
>>> from enum import Enum
>>> class A(Sized, Enum): pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases

The error is that type(Enum) is EnumMeta and type(Sized) is ABCMeta and they are unrelated.

This bug has at least the following implications:

• It obviously implies that a: ABCMeta = str typechecks.
• It makes it harder to warn about inconsistent metaclass structure without false positives. This is why mypy only catches specific cases.
• As exemplified above, class A(str, Enum) appears to have inconsistent metaclass structure, even though this is the recommended way of making an enum of strings. It might also be the case with some other builtins.

[elazarg]

Possible solution, as discussed with @ilevkivskyi and @JukkaL in gitter: use ABCMeta.register as a decorator:

@Sized.register
class str: ...

A caveat is that this does not compile with generic type, i.e. @Iterable[str].register is a syntax error; @JukkaL suggested using @Iterable.register and requiring inference of the type parameter. (I have proposed in python-ideas adding abc.register function and a __registered__ field, which may solve the syntactic problem in the future)

[gvanrossum]

Maybe if Sequence were a Protocol this would be less of a problem?

[ilevkivskyi]

Maybe if Sequence were a Protocol this would be less of a problem?

I also think this is the right way forward. Although it will be better to first implement more advanced join for protocols, otherwise ['abc', ['ab', 'bc']] will be inferred as List[object] instead of List[Sequence[str]], one can always override this with an explicit annotation. In this particular example List[object] is probably OK. So as Jukka said on gitter, it is a bad idea to indiscriminately remove explicit protocol bases everywhere, but in some particular cases this makes sense.

[Zac-HD]

I have a similar problem in HypothesisWorks/hypothesis-python#858 - sampling from an Enum works, but Mypy complains about assert len(an_enum), because it is not Sized. Obviously this does work at runtime. A minimal reproducer is:

import enum
print(len(enum.Enum('A', 'a b c')))  # prints "3"
#  error: Argument 1 to "len" has incompatible type "Enum"; expected "Sized"

I think it belongs here, but happy to move to another issue or open on if that's more useful.

[elazarg]

@Zac-HD this seems to be a mypy-specific problem, since it works if you define the enum using the class syntax. Note that the type in the error message is incorrect - it should be a Type[Enum] or something similar.

[ilevkivskyi]

Yes, mypy has special treatment of enum definitions, so it doesn't detect it in this particular "in-place" definition.

[hauntsaninja]

This will require type system features to fix, so five years on, is better discussed at https://github.com/python/typing