Marking conda as installer for pip packages

[nehaljwani]

All of the python package recipes which have python -m pip install ... or just pip install ... create one problem. They install a file by the name 'INSTALLER' in the dist-info directory. For example:

$ cat /Users/nwani/m3/envs/test/lib/python3.7/site-packages/dbf-0.97.11.dist-info/INSTALLER
pip

This is bad, because the from a user's perspective, when an end user does: conda install dbf this package is installed via 'conda' and not 'pip'. The upcoming release of pip will have a new function was_installed_by_pip(): This is going to be used for not telling the user to pip install --upgrade pip if pip is not installed via pip. I was thinking of extending it to all packages not installed by pip pypa/pip#5605 .

So, what would you recommend, conda-forge? Should we:

• stop using pip install and use the old --single-version-externally-managed
• start doing echo conda > /path/to/pkg-pkgversion.dist-info/INSTALLER
• add a patch to pip which adds an argument to mark the installer
• delete the INSTALLER file?

[nehaljwani]

fedora package maintainers already do this (for the pip package itself): https://src.fedoraproject.org/rpms/python-pip/blob/f28/f/python-pip.spec#_360

[jakirkham]

Whatever we decide, it would be nice to bake it into some sort of {{ compiler("python") }} syntax to simplify maintenance and be prepared for the next change we will inevitably make to Python packages.

[ChrisBarker-NOAA]

This is tricky, as people use conda and pip together. So when someone "pip installs" something, we DO want pip to think that any already-there dependencies were installed by pip (or, I guess, just that they are installed). Which is why we've been installing stuff with pip in conda-build.

also -- the theory is that pip might install packages that are built with tools other than setuptools -- so we can have one front end to all python package builders/installers.

So if pip does grow a way to indicate whether a package is being managed by pip, it would be really nice if it could still install a package, but optionally say "don't try to manage this one". SO maybe a PR to pip is in order. (or a post-install hack)

Of course making sure pip knows it is not managing itself is a good idea -- that could get ugly.

and I do like @jakirkham's idea of a generic python "compiler" that could be managed in one place as these things change -- could we add a {{ installer("python") }} instead?

-CHB

[nehaljwani]

This is tricky, as people use conda and pip together. So when someone "pip installs" something, we DO want pip to think that any already-there dependencies were installed by pip (or, I guess, just that they are installed). Which is why we've been installing stuff with pip in conda-build.

If I understand correctly, at the present moment (v10.0.1), regardless of what value INSTALLER holds, pip can still upgrade/downgrade/uninstall a package that was installed with the .dist-info (wheel?) directory structure or with the .egg-info (egg?) directory structure (if not installed via distutils)

So if pip does grow a way to indicate whether a package is being managed by pip, it would be really nice if it could still install a package, but optionally say "don't try to manage this one". SO maybe a PR to pip is in order. (or a post-install hack)

Yes! I'd very much like it to be part of pip itself and not a post-install hack. Nick and Donald have shared some ideas at https://github.com/pypa/pip/issues/5605 . If you have some additional thoughts regarding it, could you please share them there as well?

[nehaljwani]

cc @jjhelmus @mingwandroid @msarahan @kalefranz

[jjhelmus]

I would be in favor of recording conda or something similar in the INSTALLER file.