search

pypa / packaging.python.org

Python Packaging User Guide
http://packaging.python.org
1.39k stars 819 forks source link

How to deprecate a python package? #883

Open naveen521kk opened 3 years ago

naveen521kk commented 3 years ago

I don't know the exact procedure to do "deprecate a python package" and I couldn't find any in the docs. I am from Manim's team, we previously published under the name of manimce and later changed to manim, because we didn't have access to manim name on PyPi initially (https://github.com/pypa/pypi-support/issues/450).

Now, we have done many releases on manim but left manimce just as is. I am wondering whether I should delete it(can it create problems?) or publish a new version that explicitly fails on install and point to the new package. Or simply leave it as it is now. I would also like to know what other packages do in this case?

I don't know the exact guidelines on how to, and I would like it to be in part of https://packaging.python.org.

pradyunsg commented 3 years ago

Failing on install won't help, since the pip's dependency resolver will backtrack to an older version that doesn't fail. For your use case, I'd recommend making a new release, which fails on import with a message to use manim instead of manimce.

pradyunsg commented 3 years ago

@webknjaz I've relabeled this, since this definitely should not be a tutorial. A discussion/guide is more appropriate, talking about what the tradeoffs are between the various choices a user has.

webknjaz commented 3 years ago

Honestly, the Python Packaging ecosystem could benefit from having some sort of Obsoletes: field in the metadata which in conjunction with Provides: could be a good starting point for implementing alternatives in the installers...

Right now, you can upload a now .post0-release (see PEP440) with the readme/long description updated with instruction pointing at where to obtain the updates + maybe the runtime check wouldn't hurt. I think I wouldn't go for raising an exception but instead, I'd issue a DeprecationWarning. This sort of warnings doesn't show up by default but users who set up their dev envs with -W error and their pytest configs with filterwarnings = error will definitely notice this in their CIs. Then, you could add a UserWarning in .post1 — this one is actually printed out to stdout so it'll get logged in the end-users' terminals. And only after this consider raise RuntimeError('Go get a new version at ... or pin yours explicitly'). Allow for a few weeks/months between these post-releases.

Also, PyPI-wise, maybe you'd want to use the "yank" feature which will add a "discouraged" label on releases but the end-users will still be able to install those versions if they are requested explicitly via pins (or when they use ancient pip versions).

Also, note that because pip does not have a concept of transactions, it's best if you don't have the importable name the same (well, any contents actually) in both distributions because it'll cause problems during upgrades from one to another.

naveen521kk commented 3 years ago

Failing on install won't help, since the pip's dependency resolver will backtrack to an older version that doesn't fail.

Oh, I thought the other way round. So, failing on install shouldn't work.

For your use case, I'd recommend making a new release, which fails on import with a message to use manim instead of manimce.

Won't it create conflicts with package contents? We haven't changed the module name though, it's always manim so having an __init__.py from both packages may create problems?

Right now, you can upload a now .post0-release (see PEP440) with the readme/long description updated with instruction pointing at where to obtain the updates + maybe the runtime check wouldn't hurt. I think I wouldn't go for raising an exception but instead, I'd issue a DeprecationWarning. This sort of warnings doesn't show up by default but users who set up their dev envs with -W error and their pytest configs with filterwarnings = error will definitely notice this in their CIs. Then, you could add a UserWarning in .post1 — this one is actually printed out to stdout so it'll get logged in the end-users' terminals. And only after this consider raise RuntimeError('Go get a new version at ... or pin yours explicitly'). Allow for a few weeks/months between these post-releases.

:+1: this looks good to me. Should the DeprecationWarning and UserWarning go to setup.py or when the user imports the module?

webknjaz commented 3 years ago

Won't it create conflicts with package contents? We haven't changed the module name though, it's always manim so having an __init__.py from both packages may create problems?

This https://github.com/pypa/pip/issues/8509 will happen. You need to tell them to install the new dist in a fresh virtualenv or do pip uninstall -y manimce && pip install manim.

this looks good to me. Should the DeprecationWarning and UserWarning go to setup.py or when the user imports the module?

In runtime. setup.py is executed during build time. It won't run if the users install from wheels. And if they install from sdist and there's a wheel package present in that env, pip will cache that and any subsequent installs will just unpack that cached wheel w/o attempting to build it from the source.

naveen521kk commented 3 years ago

I did the first step of releasing a .post0 release now with a DeprecationWarning and a note on README.

dayvheed commented 3 years ago

Hi @naveen521kk. I want to help in writing the documentation for the deprecation steps you have taken regarding manimce and manim. Before I get started, it would be great to have a rounded understanding of the salient points regarding this deprecation.

I can then write the documentation. Could you put me through, please?

naveen521kk commented 3 years ago

Before I get started, it would be great to have a rounded understanding of the salient points regarding this deprecation.

We at first when creating the first release of the project; we wanted to publish to the PyPI package manim. But that was blocked because someone has uploaded malware in the same name, previously. We contacted PyPA support for getting that name unblocked so that it allows us to push, also we didn't want to delay the release because of this, that's where manimce came. We had published around 2 release in manimce and then we got the manim name back, and we changed our future releases to manim and we wanted to depreciate manimce now, so is this issue.

As a first step following what @webknjaz told in https://github.com/pypa/packaging.python.org/issues/883#issuecomment-820467146, I have published a 0.1.1.post0 release, with changing the readme and also issue a DeprecationWarning on import of the module.

dayvheed commented 3 years ago

Ok. I will come up with something and show you!

dayvheed commented 3 years ago

Hi @naveen521kk , @webknjaz

I just wrote a deprecation documentation and initiated a pull request. You may find it in #884. Please let me have your thoughts.

dayvheed commented 3 years ago

@naveen521kk @webknjaz please let me know when you've checked my PR. I really would value your feedback

naveen521kk commented 3 years ago

It's been around a week and now I pushed a .post1 with a UserWarning. After around two weeks, I will do a release that will error out on import.

dayvheed commented 3 years ago

Alright

naveen521kk commented 3 years ago

Today I pushed 0.1.1.post2 to PyPI which raise an exception on import.

So, this issue is fixed. Maybe I will keep it open until @dayvheed comes with a guide for that :).

hollabaq86 commented 1 year ago

Hi folks,

Dropping into this issue to add a +1 for better documentation on yanking a package. Here's a couple stories I'm thinking of:

As a maintainer of a python library that's hosted in PyPi, I want to know:

As a consumer of a python library that's hosted in PyPi, I want to know:

I'm happy to write the docs for this, but I will definitely need review from folks with deeper knowledge for accuracy as I've never yanked a package before (but am now considering doing so for some versions and was searching for the docs on it 😄 ).

What's the best location in pypi docs to add this documentation? Or would it be better to add this to a different library's docs such as the packaging library's docs? I want to make sure I'm adding documentation in all the right places, but I'm not deeply versed in the python packaging ecosystem. Thanks!

hollabaq86 commented 1 year ago

Another question for the group: are there alternatives for yanking a package? Like, I want a consumer of my library to still be able to download a package but they receive a warning it's deprecated and should update. Is there a feature like this for pypi hosted packages? If I should ask this Q in a different repo please let me know, I'm just trying to grok python's packaging ecosystem.