search

pypi / warehouse

The Python Package Index
https://pypi.org
Apache License 2.0
3.54k stars 952 forks source link

Support for nightly releases in pypi #8792

Open omry opened 3 years ago

omry commented 3 years ago

I would like to open a discussion around what might nightly build support in pypi look like. First, a small IRC paste:

omry> What's the general take on publishing nightly releases to pypi? the package is not large (about 200kb now), but I am not sure if things like maintaining last X nightlies are doable (nuke old nightly as if they never existed). mgedmin> my gut feeling is "please don't", but I'm not a pypa member PSFSlack5> omry: If the project is small, it's not really a big deal. omry> PSFSlack5: my concern is more about polluting the version name-space. is there a programmatic way to delete packages? PSFSlack5> omry: There is not. Some projects do a separate project name entirely, like https://pypi.org/p/tf-nightly omry> PSFSlack5, that's not a bad solution. It would be nice to have a programmatic interface to manipulate packages for cases like this. omry> in the tf example, they upload about 1GB a day: https://pypi.org/project/tf-nightly/2.4.0.dev20201018/#files omry> I don't think they want to keep old nightlies forever. cooperlees> omry: Prob worth an issue / discussion @ https://github.com/pypa/warehouse/issues cooperlees> I'd love to see nightlies be able to only keep the last x weeks of nightlies for example and stable releases. Would be helpful for bandersnatch / mirror peoples

Something that could be nice is a way to maintain X last nightly packages automatically and have older nightlies be automatically deleted.

The pip client can potentially provide a specific error for people who are trying to install deleted nightly packages, something like:

The nightly package foo==version expired. 
Please install a newer nightly package or switch to a stable release.
nightly versions: [newest K nightly versions]
stable versions: [newest K stable versions]

Pip should also not show nightly packages in the message it normally shows when there is a mismatch version number.

faxu commented 3 years ago

+1 for this feature. This has become troublesome for our project esp with the pypi size limit. Manually deleting all older packages every [n] weeks is difficult to manage.

balopat commented 3 years ago

+1 for this feature. We find it very nice to be able to publish to a "latest" channel just by providing pip install --pre <package> (compared to having to provide a separate index, separate infra for publishing dev versions) and are in the process of switching over to that model as it is not explicitly discouraged anywhere. However, there aren't many projects doing this, and there isn't a good retention policy which created a bit of hesitation on our end to pull the trigger. As our project is tiny, this shouldn't matter much, we'll be below the project size limit.

Currently we have the two different packages like the tensorflow ecosystem. However, that creates a namespace collision. If you install accidentally both packages (for which there is no way to automatically defend against using only pip), the environment can get into a bad state that can get only resolved by uninstalling both packages.