search

pimutils / khal

:calendar: CLI calendar application
https://lostpackets.de/khal/
MIT License
2.63k stars 208 forks source link

Should you really use /usr/bin/env? Plays badly with conda #1202

Open hamdav opened 2 years ago

hamdav commented 2 years ago

I installed khal in arch linux (simply sudo pacman -Syu khal) and when I ran khal in the terminal it gave a ModuleNotFoundError: No module named 'khal' for the line from khal.cli import main_khal in /usr/bin/khal. After a little bit of debugging it turns out that installing khal installs it in the system python, not the conda python that I use for my own code (as it should) but writing the shebang #!/usr/bin/env python causes /usr/bin/khal to execute with my conda python, since that is what is first in my path. Maybe you should change this to #!/usr/bin/python to use the system python instead. That worked for me at least.

mschilli87 commented 2 years ago

IMHO if /usr/bin/env python doesn't point to the 'correct' Python path, that's a problem with your system configuration. Hard-coding it to /usr/bin will break it on all system installing Python elsewhere but properly configuring the PATH environment variable accordingly.

WhyNotHugo commented 2 years ago

After a little bit of debugging it turns out that installing khal installs it in the system python, not the conda python that I use for my own code (as it should)

You installed via pacman, pacman has no knowledge of your conda environment.

writing the shebang #!/usr/bin/env python causes /usr/bin/khal to execute with my conda python, since that is what is first in my path. Maybe you should change this to #!/usr/bin/python to use the system python instead. That worked for me at least.

Actually, we can't hardcode /usr/bin/python because this might vary depending on the distribution (e.g.: /usr/local/bin/python3 on openbsd, or /usr/bin/python3 on alpine). Your idea is valid here, but I'm not sure that there's any mechanism in pip that would let us accomplish that.

Do any other packages work around this issue? It feels like something that needs to be resolved downstream.

mschilli87 commented 2 years ago

@WhyNotHugo

Do any other packages work around this issue?

I only know this issue from packaging for GNU Guix where the 'solution' is to 'fix' the shebangs by hardcoding with Guix's version as part of the build/install process.

It feels like something that needs to be resolved downstream.

I agree that this is a problem 'caused' by conda (or rather mixing it with system pip) that doesn not need any special treatment by khal.

What makes me thinks so is that I am pretty certain that if @hamdav would take a clean (vanilla) Arch Linux installation and run

sudo pacman -Syu khal

everything would work fine. The issue is that once conda installed (and 'loaded'), it overwrites the system python breaking all non-conda Python scripts that don't hardcode /usr/bin/python in the shebang (for the reasons @WhyNotHugo mentioned). I am not aware of a good solution here which IMHO is simply a limitation of Python's module/package system.

WhyNotHugo commented 2 years ago

Actually, the khal binary we ship is a one: https://github.com/pimutils/khal/blob/master/bin/khal

IIRC, the one setuptools auto-generates does a lot of stuff and is too slow at startup.

However, package maintainers copy this themselves (e.g.: https://github.com/archlinux/svntogit-community/blob/packages/khal/trunk/PKGBUILD#L47 ). I guess we could hint at package maintainer to patch this with a hardcoded shebang?

hamdav commented 2 years ago

that's a problem with your system configuration

but I want to get the conda python when I type python in the terminal, right? And (as far as I understand) /usr/bin/env python is always the same as the thing I get when I type python in the terminal.

You installed via pacman, pacman has no knowledge of your conda environment.

Yes, I know, sorry, after rereading I realize the (as it should) was not very clear but I meant that khal indeed should be installed on the system path. And run with the system python.

WhyNotHugo commented 2 years ago

but I want to get the conda python when I type python in the terminal, right? And (as far as I understand) /usr/bin/env python is always the same as the thing I get when I type python in the terminal.

conda changes you $PATH. Activate a conda environment and check the output of which python.

WhyNotHugo commented 2 years ago

Strictly speaking, this is not a bug for khal nor does khal do anything wrong.

I do agree that this could be improved.

mschilli87 commented 2 years ago

@hamdav:

but I want to get the conda python when I type python in the terminal, right?

I don't know what you want. Python was note developed with the idea of several incompatible versions coexisting on the same system. It's fast development and the big number of new packages depending on it combined with the comparable slow adaptation by many distributions have led us to a situation where this is a reality now though. AFIAC, your system Python is (and should be) the default on your system. If you want to run something that requires a different version of Python, this should be explicit. I don't know why you have the conda version of Python on your system. If it is to run the latest version of some package not even available in the AUR or to overcome incompatiblites with other Python-packages requiring a different version, I would activate the conda environment in the shell session I start that program from (locally!) only. That's how I handle these kind if situtation with GNU Guix, I never have Python on my default profile but rather use the one provided by my host OS (in my case Gentoo). I understand your situation and agree that changing the shebang of khal would fix it for you, but this has to be done at a point where the exact location of the system installation of Python is known. Since khal can be intalled on many systems with different default locations for python, this cannot be done here, IMHO. Even searching your system during install is not guranteed to work: Depending on whether you install khal or Python via conda first, khal could pick up on it or not. So AFAIC the only place this could be handled at all would be on the Arch side, since on evey Arch install, python should be located in /usr/bin. So they could replace #!/usr/bin/env python by #!/usr/bin/python in every *.py file as part of there installation process to ensure Python-reliant packages installed by pacman always use the Python interpreter installed by pacman, even if a conda environment (or Guix profile) providing an alternative Python installation was loaded. Note however, that there might be other Arch users out there that want to overwrite which Python version is used for khal by PATH (e.g. to test a new version of Python via a conda environment against all there Python-dependent tools without touching the (fallback) system installation). Fixing your isue on the Arch level would break their use case. If I was maintaining a distribution, I would not change package definition to fix issues caused by installing software using third party packaga providers like conda, Guix, snap, etc. for that reason but I don't the Arch maintainers' take on this.