Introduce an Unknown type

[Eyal-Shalev]

Unknown Definition

A type that can only be cast to Any

Problem

Currently when type checkers fail to understand the type of a function/variable, they fallback on Any.

T = TypeVar("T")

def foo(fn: Callable[[], T]) -> T:
  return fn()

bar = foo(lambda: 5)

# The type of bar will fallback on `Any` so the type checker will not warn on misuse
_ = bar[0]

Suggestion

If an Unknown type is introduced, and can be defined as a fallback in the checker then all operations performed on it (that assume a specific type) will fail.

T = TypeVar("T")

def foo(fn: Callable[[], T]) -> T:
  return fn()

bar = foo(lambda: 5)

# The checker will mark this as an error because subscripting is not defined on 
`Unknown`.
_ = bar[0]

Inspirations

Typescript: https://www.typescriptlang.org/docs/handbook/type-compatibility.html#any-unknown-object-void-undefined-null-and-never-assignability

any and unknown are the same in terms of what is assignable to them, different in that unknown is not assignable to anything except any.

unknown and never are like inverses of each other. Everything is assignable to unknown, never is assignable to everything. Nothing is assignable to never, unknown is not assignable to anything (except any).

[srittau]

There already is an "unknown" type in Python: object. JavaScript - and by extension Typescript - doesn't have a universal base type by virtue of having scalar types, making a synthetic unknown type necessary.

That said, I think some type checkers already have an internal "unknown" type although that has a slightly different purpose.

[erictraut]

As @srittau said, the object type serves a similar purpose in Python's type system.

Pyright implements an Unknown type to distinguish between an explicit Any (one that comes from an Any type expression) and an implicit Any (one that is generated by a type checker when a type is unspecified, a TypeVar cannot be solved, an import cannot be resolved, etc.). For details, refer to the pyright documentation.

Pyright's concept of Unknown differs from TypeScript's unknown in that: 1) Unknown cannot be used in a type expression because it is always implicit and 2) assignability rules for Unknown are the same as Any.

Here's a sample in pyright playground

[Eyal-Shalev]

As @srittau said, the object type serves a similar purpose in Python's type system.

Pyright implements an Unknown type to distinguish between an explicit Any (one that comes from an Any type expression) and an implicit Any (one that is generated by a type checker when a type is unspecified, a TypeVar cannot be solved, an import cannot be resolved, etc.). For details, refer to the pyright documentation.

Pyright's concept of Unknown differs from TypeScript's unknown in that: 1) Unknown cannot be used in a type expression because it is always implicit and 2) assignability rules for Unknown are the same as Any.

Here's a sample in pyright playground

1. Thank you. I never checked Pyright, so wasn't aware it is so much better than MyPy 🙇 - I'm going to try and move my team from MyPy to Pyright.
2. I think that having the unknown type as a PEP (and thus in the reference implementation i.e. MyPy) is very valuable.

[vtgn]

I don't find the use of "object" logical and it can be disturbing for the developers. The types parsers use the word "Unknown" already when they don't know the type of a variable, and not "object". Creating the type "Unknown" is much more appropriate in every cases.

[srittau]

I don't think there's much value in creating a second "unknown" type that's virtually equivalent to object. As pointed out by erictraut, the unknown type used internally by pyright has different semantics than what's proposed here.

[vtgn]

I don't think there's much value in creating a second "unknown" type that's virtually equivalent to object. As pointed out by erictraut, the unknown type used internally by pyright has different semantics than what's proposed here.

object is absolutely not equivalent to Unknown, because object type declares several properties and methods that don't exist on values like None, int, str. So if you have:

o: object = 3
print(o.__dict__) # => no problem for static typing, it exists for object type

but at the execution : AttributeError: 'int' object has no attribute 'dict'. Did you mean: 'dir'?

It's totally wrong to say that object is an equivalent of Unknown, because it is clearly NOT as proved above! An Unknown type is absolutely necessary to force the developer to check the contents of the value before to call a property/method on it.

[srittau]

>>> isinstance(3, object)
True
>>> isinstance(None, object)
True

That not all types have a __dict__ attribute is true, and unfortunately not representable using the type system. But this is unrelated to None, int and other builtins to being "special". The same is true for other types implemented in extension modules:

>>> import re
>>> re.compile("").__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 're.Pattern' object has no attribute '__dict__'. Did you mean: '__dir__'?

[vtgn]

>>> isinstance(3, object)
True
>>> isinstance(None, object)
True

That not all types have a __dict__ attribute is true, and unfortunately not representable using the type system. But this is unrelated to None, int and other builtins to being "special". The same is true for other types implemented in extension modules:

>>> import re
>>> re.compile("").__dict__
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 're.Pattern' object has no attribute '__dict__'. Did you mean: '__dir__'?

OMG!!! For me, this is clearly a huge inconsistency of the Python langage, violating the most basic rules of static typing. >___<° !!!! I understand it should have been hard to add static typing in a language who hadn't, but that choice was a huge mistake. What a shame!

[srittau]

One thing we could try is adding __hash__: None annotations to such classes, similar to what we do with unhashable classes (where we use __hash__: ClassVar[None]).

[vtgn]

@srittau I understand it is hard to fix this kind of type's inconsistency because of historical features of Python. :(

The object class is a parent class of every objects, but its declared members are not inherited by all the sub-classes. :/ In practical terms, this does not completely violate the rules of inheritance, because it amounts to declaring a member in a parent class, which is redefined as throwing an exception in a child class. It is not clean, but it is correct.

Even if we don't use the object type in our code, the fact that all classes inherit from it indicates automatically that its members are available for all classes, so it doesn't fix the problem.

To fix this problem without developing a Python 4 version implementing a proper typing overhaul, the static typing tools should ignore the object class declared members. I don't know if there are other builtins classes having such a problem of declared members not inherited by sub-classes, but the tools should take into account all of them to only keep the existing ones.

I see no other simple solution... :/