[FYI] recently uploaded PyPi packages don't have short package links

[frol]

The packages used to be under ../../packages/source/<first letter>/<pkg name>/, now only available as ../../packages/<two-char hash>/<two-char hash>/<long hash>/. The according links also changed in the "simple" index page. This has changed the disk layout of the bandersnatch. Our build system broke due to the mirror of bandersnatch doe not have the same disk layouts in the disk.

Related issues:

• https://sourceforge.net/p/pypi/support-requests/626/
• https://bitbucket.org/pypa/pypi/issues/447/direct-links-of-packages-gone

Example:

There is a package colorlog with two versions: 2.6.1 (old) and 2.6.3 (new). The following link works fine:

https://pypi.python.org/packages/source/c/colorlog/colorlog-2.6.1.tar.gz

However, if you just change the version number, you will get 404:

https://pypi.python.org/packages/source/c/colorlog/colorlog-2.6.3.tar.gz

The working link to 2.6.3 version is:

https://pypi.python.org/packages/cd/e7/7e14ce72038e83bc475a1f708485e5ea4a789beef26ff64b732918f860e6/colorlog-2.6.3.tar.gz

[frol]

Hmm, the solution turned out to be quite simple. We should use https://pypi.io/ instead of https://pypi.python.org/

[ocefpaf]

There is a new link to the "new" PyPI (warehouse) that is something like this:

https://pypi.io/packages/source/c/colorlog/colorlog-2.6.3.tar.gz

[ocefpaf]

@frol you were faster than me :wink:

[jakirkham]

Yeah, I think we also have a weird mix of URLs in our ecosystem that are going to need to be updated. Would be nice to have a script to do.

Right @pelson? 😉 Just asking as I know you have more experience with doing these sorts of global changes.

[grlee77]

Also, IIRC @pelson had expressed preference for links to github over pypi. It would be good to get more feedback on whether that is the general consensus going forward for new PRs to conda-forge. And, if it is, should I update the links in my recently merged feedstocks or just wait until a future release change to make the change?

[pelson]

There is no rush on this. Basically, we will always be in a situation where our recipes reflect how we previously built a distribution, rather than be an always up to date source of how to build a distribution now. We should gradually fix this as time goes on (and perhaps I will put together a script which pushes this along for the last stragglers in a month or two).

For now, update as necessary, but you are in control - there is nothing stopping you from updating all of your feedstocks if you are so inclined.

I agree about the GitHub vs PyPI advice - basically @jakirkham and I have a minor disagreement (a healthy one), it is something we can talk about over the next few weeks. I hope we can provide clearer guidance in the future.

[ocefpaf]

I agree about the GitHub vs PyPI advice - basically @jakirkham and I have a minor disagreement (a healthy one)

Here is my 2 cents:

I liked the PyPI URLs due to laziness: in 1 copy-and-paste pass I had the source and the md5 info for the recipe. Now that we have to edit the PyPI URL I am tempted to go straight to GitHub.

Of course there are times when we don't have a choice, like when:

• there are no tags on GitHub;
• PyPI is updated and GitHub is not.

The issue arises when there is a choice. And is more important when the two sources are different. For example: some PyPI source distributions bundle more than just the source. In that case @pelson votes for the hard road to build everything and avoid the bundle. I agree, but I am not against being lazy and postponing that just to get a package in the channel.

[jakirkham]

See the comment above if you have issues with your Python URLs, @183amir.

[jakirkham]

cc @wesm @mariusvniekerk

[jakirkham]

So, I think we know now how to do these links. Many examples exist in the feedstocks. Also the example recipe has been updated to use this too. Going to go ahead and close this out.