search

pytorch / data

A PyTorch repo for data loading and utilities to be shared by the PyTorch domain libraries.
BSD 3-Clause "New" or "Revised" License
1.12k stars 149 forks source link

Include `py.typed` marker file for mypy type hint compatibility #779

Open kiukchung opened 2 years ago

kiukchung commented 2 years ago

🚀 The feature

From what I can tell torchdata actually had type stubs but mypy won't pick it up because it is missing the py.typed marker file (see: https://mypy.readthedocs.io/en/stable/running_mypy.html#missing-library-stubs-or-py-typed-marker)

The ask is to add py.typed to the package data similar to how torch does it so that projects using torchdata can have mypy recognize torchdata modules type hints.

Motivation, pitch

My project uses torchdata and also uses mypy for static type checking. Since mypy does not pick up torchdata's type stubs I get an error as:

mfive/datapipes/amzn/datakit.py:9:1: error: Skipping analyzing "torchdata.datapipes": module is installed, but missing library stubs or py.typed marker  [import]

Alternatives

I could add a ignore-missing-imports in my mypy.ini file:

[mypy-torchdata.*]
ignore_missing_imports = True

But that means that all the torchdata related types will be typed as Any, hence not letting me enable checks like disallow_subclassing_any=True if I'm writing my own custom datapipe that subclasses torchdata.datapipes.iter.IterDataPipe.

Additional context

No response

ejguan commented 2 years ago

Sounds like a reasonable request. Sending a patch shortly.

kiukchung commented 2 years ago

Awesome, I can confirm mypy is able to get type hints for torchdata from my project. Will depend on the nightly until a dot release. Any ideas on when 0.4.2 will be released?

ejguan commented 2 years ago

@kiukchung I am sorry that I have to revert the PR until we fix the wrong typing in torchdata. See: https://github.com/pytorch/data/issues/782

And, in terms of the official release, it should be happen in the a 1/2 months aligned with PyTorch Core.