stubtest: False-positive errors for `type[T]` attributes with non-`type` metaclasses

[AlexWaygood]

Bug Report

Given a stub like so:

from abc import ABCMeta
class Y(metaclass=ABCMeta): ...
class Z:
    foo: type[Y]

And a runtime like so:

from abc import ABCMeta
class Y(metaclass=ABCMeta): ...
class Z:
    foo = Y

Stubtest will issue the following complaint:

error: Z.foo variable differs from runtime type abc.ABCMeta

This error can be reproduced with any custom metaclass; it's not just ABCMeta that triggers the bug:

# STUB
class Meta(type): ...
class Y(metaclass=Meta): ...
class Z:
    foo: type[Y]

# RUNTIME:
class Meta(type): ...
class Y(metaclass=Meta): ...
class Z:
    foo = Y

To Reproduce

1. Check out a local clone of mypy

2. Activate a virtual environment with an editable install of mypy

3. Apply this diff:

   diff --git a/mypy/test/teststubtest.py b/mypy/test/teststubtest.py
   index 2adbfaac2..60ddebc7f 100644
   --- a/mypy/test/teststubtest.py
   +++ b/mypy/test/teststubtest.py
   @@ -208,6 +208,21 @@ class StubtestUnit(unittest.TestCase):
            """,
            error="X.mistyped_var",
        )
   +        yield Case(
   +            stub="""
   +            class Meta(type): ...
   +            class Y(metaclass=Meta): ...
   +            class Z:
   +                foo: type[Y]
   +            """,
   +            runtime="""
   +            class Meta(type): ...
   +            class Y(metaclass=Meta): ...
   +            class Z:
   +                foo = Y
   +            """,
   +            error=None
   +        )

4. Run pytest ./mypy/test/teststubtest.py

Expected Behavior

No error should be emitted.

Actual Behavior

An error was emitted.

Cc. @hauntsaninja

[AlexWaygood]

The issue seems to be that the call to is_subtype_helper here returns False, whereas for a class without a metaclass, it returns True: https://github.com/python/mypy/blob/d27bff62f71a8b914b1df239467148e81d2e88a2/mypy/stubtest.py#L887

[hauntsaninja]

So the question is_subtype_helper gets posed is whether Meta is a subtype of type[Y]. This fails for the same reason that mypy will complain about x: type[Y] = Meta. The fix is probably some change in get_mypy_type_of_runtime_value — we currently always make Instances, but maybe we should have some complicated logic that comes up with TypeType for metaclasses? Might be a little tricky to get heuristics right, but subclass of type is hopefully good enough