search

pypi / warehouse

The Python Package Index
https://pypi.org
Apache License 2.0
3.54k stars 953 forks source link

typeshed and third-party packages #4967

Open srittau opened 5 years ago

srittau commented 5 years ago

Quick background: typeshed is a collection of PEP 484 type hint files for Python's standard library as well as third-party libraries that don't ship their own type hints. Currently, type checkers such as mypy and pytype as well as IDEs such as PyCharm ship a complete copy of typeshed. Of course, this does not scale well and is fairly inflexible.

Therefore, we are currently discussing (in python/typeshed#2491) to take a similar route to typescript's DefinitelyTyped project: typeshed aims to become a central repository for high-quality type stubs for projects that don't ship their own. third-party stubs are automatically built and uploaded to pypi - one PEP 561 stubs-only package per upstream package - when they change in the repository.

It is of course highly desirable for those stubs to follow a consistent naming scheme. In typescript, for a packages foo, you know you can just install the package @types/foo to get appropriate high-quality hints from DefinitelyTyped. Our currently plan is to use foo-ts as a package name for an upstream package called foo, but we are absolutely not married to that scheme.

One issue we are worried about is possible name squatting. Since the whole point is that package names are predictable, it is very easy to squat the names of successful packages that don't have type hints, yet. This could potentially be harmful if potentially dangerous code is uploaded under a squatted, since I expect people to blindly install foo-ts in the future if typeshed becomes successful. (I know I do with DefinitelyTyped packages.)

We appreciate all feedback and input on this, either here or in python/typeshed#2491. If another venue is a better place to raise this issue, please let me know.

dstufft commented 5 years ago

I would suggest doing something like ts.foo or even typeshed.foo or types.foo or whatever. The reason for putting the commmon prefix first, is that I have plans to allow projects to claim a namespace, and I've always planned on making that so that it was anchored to the start of the name. My plan has been to implement something like Nuget's ID Prefix Delegation.

srittau commented 5 years ago

(#4164 is also relevant to this discussion.)

dstufft commented 5 years ago

Yea a lot of my comments in #4164 are directly relevant, because I originally understood #4164 to be proposing what this ticket is proposing.

srittau commented 5 years ago

Having namespaces would solve this problem very neatly, of course. Does pypi already support uploading packages containing a dot?

ethanhs commented 5 years ago

I agree a prefix of ts.pkg or typeshed.pkg make much more sense here. In #4164, and in PEP 561, the idea was to make the stubs package a related package to the main one, just like pytest-xdist is a related package to pytest. Here we want to signify the packages are related to the typeshed project, so some prefix based on typeshed makes the most sense to me.

EDIT: also yes, you can upload packages with a dot @srittau

dstufft commented 5 years ago

Yea, it actually considers -, ., and _ to all be equivalent so ts-pkg and ts.pkg are the same thing. I just used . out of habit. (well and because I have a slight preference for using it for namespaces). But they're all equal as far as any standards based packaging tool is concerned.

So there currently is not a mechanism for registering namespaces. There are a few ways this could go down.

The "best" way is for someone who has the time to implement namespace reservations, then this feature is effectively just someone associated with the typeshed project has to pick their namespace and register it.

However, that obviously requires a decent amount of effort in a project that isn't Typeshed, so I can understand if that isn't attractive to typeshed developers. The other real option is since this namespace would be sort of a semi-standard thing (as opposed to something entirely related to a third party thing) is that we could hardcode in the chosen namespace. The big downside to that is you wouldn't be able to register new packages within that Namespace, you'd have to ask for an admin to register the name and then transfer it to you first.

srittau commented 5 years ago

I will look into the namespace thing, hopefully in the next few months as I consider this a showstopper for typeshed going forward. No promises. Is #2589 and/or distutils-sig the best place to discuss namespaces?

dstufft commented 5 years ago

Hmm, I think that #2589 is asking for a different thing, it wants to force everyone into some user namespace, and allow some vote or something to promote a project into the "flat" namespace. I'm on my phone, but I think we should just open an issue to copy Nuget's prefix id reservations system (or turn this into into it).