jhonatan-lopes commented 2 years ago

The bug

657 introduced a `py.typed` marker to indicate to mypy that the package is typed. However, the built distribution on pypi (wheel file) is missing the `py.typed` file, whereas the source distribution is including it, as it should.

When building the package locally with pip wheel . (with my fork of pdfplumber), the package builds successfully and py.typed is included in the build and the wheel file.

Expected behavior

The py.typed file maker should be included in the pypi wheel distribution.

Actual behavior

The py.typed file maker is missing from the built distribution on pypi.

Screenshots

First, the .tar.gz source distribution showing the py.typed file:

Now, the .whl built distribution missing the py.typed file:

The locally built .whl distribution:

Environment

pdfplumber version: 0.7.4
Python version: 3.10
OS: Manjaro Linux

jsvine commented 2 years ago

Hi @jhonatan-lopes, and thanks for raising this issue. I'm not much of a packaging/distribution expert, so I have a couple of questions to help clarify my understanding:

It makes sense to me that the source distribution would/should have py.typed. But why would/should the built distribution have it? My understanding is that py.typed is primarily useful for working with the source files, but I may simply be misunderstanding.
To build the distributions, I run python -m build (based on these recommendations). Is that also what you're running? Or a different command?

jhonatan-lopes commented 2 years ago

Hi @jsvine! I'm not a distributions expert in any way either :) but I'm trying my best to get typing annotations working for this amazing library, especially given the trouble with putting up the annotations in the code in first place. Packaging should have been the easy part.

To your questions:

py.typed is needed whenever you install the library. mypy sees this file and interprets that as "yes, this library has typing annotations because I can see the py.typed file". Even if all the code has type hints, mypy will throw an error for the library if it doesn't find stubs or a py.typed file. Now, when installing pdfplumber from the source distribution, this file is included with the library and mypy sees that the library has type hints. With the built distribution, however, the file is not being included, so mypy throws an error for the whole library because it doesn't find the py.typed file or stubs.
I tried to do python -m build but I got an error (FileNotFoundError: [Errno 2] No such file or directory: '/tmp/build-via-sdist-ys6fk6ba/pdfplumber-0.7.4/requirements.txt') that I couldn't figure out from the source. Instead, I used pip wheel . to build it, and it builds fine, i.e. it includes py.typed on the source and built distributions.

I honestly don't know why python -m build wouldn't be including py.typed onto the built distribution and I couldn't figure it out directly from inspecting the code.

However, if I might humbly offer a suggestion, in our projects we have been migrating away from a setup.py/setup.cfg/requirements.txt setup to using poetry. It has greatly simplified dependency management, environment setup and building distributions for us. If you would like to try it, I could set it up and submit a PR. Please let me know. It has always built the right files and properly included the py.typed marker.

jhonatan-lopes commented 2 years ago

I found a similar problem on httpx package, from a while ago. Please check out this issue.

It seems that they had the same issue as it's happening here. The only difference in code now seems to be that their setup.py contains the following declaration:

package_data={"httpx": ["py.typed"]},

which is not present on pdfplumber's setup.py.

Nonetheless, I'm not sure which build system they are using and whether that would be enough to build it properly. I thought that having the include in MANIFEST.in was enough.

jhonatan-lopes commented 2 years ago

Sorry, new update. From mypy's official documentation, it seems that

zip_safe=False

must be added to setup() when using setuptools and mypy together.

The httpx package figured this out with this PR.

@jsvine, I could raise a PR with these changes if you would like to. Please let me know.

jsvine commented 2 years ago

Thank you for those explanations and that detective work, @jhonatan-lopes! And thank you for offering to submit a PR. That sounds great, and I'd appreciate it. Cheers.

jsvine commented 2 years ago

Fixed via your PR in #703 — many thanks!

jsvine / pdfplumber

pip whl missing `py.typed` #698

The bug

657 introduced a `py.typed` marker to indicate to mypy that the package is typed. However, the built distribution on pypi (wheel file) is missing the `py.typed` file, whereas the source distribution is including it, as it should.

Expected behavior

Actual behavior

Screenshots

Environment

jsvine / pdfplumber

pip whl missing `py.typed` #698

The bug

657 introduced a py.typed marker to indicate to mypy that the package is typed. However, the built distribution on pypi (wheel file) is missing the py.typed file, whereas the source distribution is including it, as it should.

Expected behavior

Actual behavior

Screenshots

Environment

657 introduced a `py.typed` marker to indicate to mypy that the package is typed. However, the built distribution on pypi (wheel file) is missing the `py.typed` file, whereas the source distribution is including it, as it should.