search

pypa / pip

The Python package installer
https://pip.pypa.io/
MIT License
9.53k stars 3.03k forks source link

RFE: Allow to --set-installer for custom INSTALLER value #8156

Open hroncok opened 4 years ago

hroncok commented 4 years ago

What's the problem this feature will solve?

In Fedora, we use pip to install wheels during the build of RPM packages. We then replace pip with rpm in the INSTALLER file to indicate: This package has been actually installed trough rpm on the user systems (despite it being installed by pip in the build system).

The hash of the INSTALLER file is recorded in RECORD. When we change the value, we should also recalculate the hash and adjust RECORD.

This is getting fragile. I'd rather tell pip to put arbitrary value to INSTALLER than sed it afterwards.

Describe the solution you'd like

We'd like to have --set-installer option (or --installer etc.) for pip install that we would use like this:

pip install --set-installer=rpm ...

It would put the value (rpm in the example) to INSTALLER instead of pip and would put the correct hash to RECORD.

Alternative Solutions

The alternate solution is what we use now + sed-ing the hash:

pip install ...
sed -i s/pip/rpm/ ...dist-info/INSTALLER
sed -iE 's|(\.dist-info/INSTALLER,sha256=)[^,]+,|\1'$(sha256sum ...dist-info/INSTALLER | cut -f1 -d" ")',|' ...dist-info/RECORD

Additional context

https://discuss.python.org/t/pep-610-usage-guidelines-for-linux-distributions/4012/17

I'm happy to submit a PR for this if agreed upon.

dstufft commented 4 years ago

I wonder if a better overall solution would be to write a simple, but lower level wheel installer.

My thinking here is basically instead of trying to shoe horn pip into this, it might be better to just make something lower level designed to be part of a build process like this. It could be laser focused on just installing wheels, so no need for network code or resolvers. If it was done as a python api with a cli interface to that api, then pip and fedora could both use it.

I dunno. My main concern here is adding a proliferation of options and then still ending up with a subpar workflow for fedoras use case. Particularly since this is not our primary use case, there is likely a fair amount of risk of new features regularly needing to be worked around ala direct url support.

Beyond that, I dont personally have a problem with this.

pradyunsg commented 4 years ago

I wonder if a better overall solution would be to write a simple, but lower level wheel installer.

That's what I imagine https://github.com/pradyunsg/installer would become eventually.

See https://github.com/pradyunsg/installer/issues/1 for context.

hroncok commented 4 years ago

I dunno. My main concern here is adding a proliferation of options and then still ending up with a subpar workflow for fedoras use case. Particularly since this is not our primary use case, there is likely a fair amount of risk of new features regularly needing to be worked around ala direct url support.

I get that and I think you are right.

:100: to @pradyunsg for the installer project :)

EpicWink commented 4 years ago

What's the purpose of INSTALLER? If it's to identify what constructed the current package layout, then should it stay as pip? If it's to let other programs know who's in control of managing the package, then it would become rpm

Interestingly, PEP 376 recommended to have an --installer option in distutils for exactly this reason

hroncok commented 4 years ago

We use it to let other programs know who's in control of managing the package. Especially for things like https://github.com/pypa/pip/issues/5346 (all other uses are strictly hypothetical).

dstufft commented 4 years ago

I get that and I think you are right.

Do you think we should close this, or do you still think you would like this feature in the absence of the installer project being functional?

hroncok commented 4 years ago

I'd still like this feature, but I have realized our RECORD file is a mess anyway, so I'll dig deeper into what needs to be done on our side to prevent that.

I plan to look closer and we might very well just need to recalculate all hashes, drop all hashes or make RECORD optional for packages managed by external tools. When we go that way, setting installer via sed will be sufficient for us.

Can we keep this open for couple weeks and I'll get back with more info?

dstufft commented 4 years ago

Sure

hroncok commented 4 years ago

I plan to look closer and we might very well just need to recalculate all hashes, drop all hashes or make RECORD optional for packages managed by external tools.

https://discuss.python.org/t/pep-376-vs-rpm-packaging-is-record-necessary/4126