Open dffffffff opened 1 year ago
pip3 install -U -r requirements.txt --break-system-packages
fix the problem, but is it safe ?
Well, --break-system-packages
makes it clear that it isn't save and now the user requested the explicit dangerous setup.
If say, a update of a package brakes a OS script/tool, it can be serious or not but how would you know after running pip3 install -U -r requirements.txt --break-system-packages
?
A proper solution would be to use a dedicated virtualenv
for LibreNMS. This would also have the benefit of protecting LibreNMS from breakage caused by other pip3 runs.
https://www.debian.org/releases/stable/amd64/release-notes/ch-information.en.html#python3-pep-668
As far as I'm concerned installing packages into the LibreNMS home directory will not break system packages, but the flag is still required there.
pip3 install --user -r requirements.txt --break-system-packages
I am not familiar with venv, so someone else will have to update docs and code to cope. And also take care not to break existing installs.
Are both --user
and --break-system-packages
needed? If packages are installed into the user context they shouldn't be able to break the system IMHO...
I have upgraded from debian 11 to debian 12.
After update Debian I have run ./validate.php and got same error.
To fix this - as root I run:
apt install python3-full
mv /usr/lib/python3.11/EXTERNALLY-MANAGED /tmp/
and as librenms I run only:
pip3 install -r /opt/librenms/requirements.txt
That fix the case.
@PVasileff that works, but it seems like a good way to break your system :D
It should be possible to run it in a venv by using the venv's Python binary itself. But you would still need to be in the correct working directory. For example, if the venv was installed in /opt/librenms/venv
, you could use the pip3 and python3 from /opt/librenms/venv/bin
. If your working directory was /opt/librenms, you could just do ./venv/bin/pip3 -install -r requirements.txt
. But obviously all Python scripts from this venv would need to be either run with sourcing the venv's activation script first or by directly calling the binaries from the venv.
I am facing this when I try to install librenms on a new system, following the official guide: https://docs.librenms.org/Installation/Install-LibreNMS
Now it’s unclear to me how to proceed.
Just tested on Debian 12:
python3 -m venv venv
source venv/bin/activate
then pip3 install -r requirements.txt
/opt/librenms/.profile
to set PATH to use venv binaries : export PATH=/opt/librenms/venv/bin:$PATH
@mairie-varces This sounds to me like the correct way to do it.
@slalomsk8er we're running librenms in a lab setup for now (trying it out!) and we're still running into surprises so my suggestion is not a fix yet, but it may help people work around the issue.
@mairie-varces I guess the path needs to be set in anything that isn't sourcing /opt/librenms/.profile but calling librenms python scripts - poller cronjobs, php-fpm, snmp-traps handler?
The validation script succeeds when run from the shell but does not when called by php-fpm via /LibreNMS/Validations/Python.php.
The python validation script is called via a symfony Process function. Is this the case for every python script used by librenms? If so I have the fix!
After setting up the venv as described in my previous post, setup php-fpm to use the correct path by adding the following line to /etc/php/8.2/fpm/pool.d/librenms.conf :
ENV[PATH] = /opt/librenms/venv/bin:/usr/local/bin:/usr/bin:/bin
As cron has a limited default path also add PATH=/opt/librenms/venv/bin:/usr/local/bin:/usr/bin:/bin
at the start of /etc/cron.d/librenms.
@slalomsk8er are you aware of any other way to launch python programs in librenms? So far, with a bit of tinkering and grep, I have identified:
@mairie-varces
Maybe via PHP trough dist/librenms-scheduler.service
but I'm not sure if a path set there will be carried though to any python programs.
Hello, I'm currently testing a version with the python venv implementation.
The version is available at : Pelt10/librenms:python_venv I'm open to any review, help...
I can open an PR, if need, to continue discussion.
On Debian 12, only command-runner
is missing. Other packages are system-wide available and installed. So this command fix the issue : pip3 install command-runner --break-system-packages
I guess that the system package won't be broken because it doesn't exist.
This issue has been mentioned on LibreNMS Community. There might be relevant details there:
This issue has been mentioned on LibreNMS Community. There might be relevant details there:
https://community.librenms.org/t/errors-validate-php/24970/2
This issue has been mentioned on LibreNMS Community. There might be relevant details there:
https://community.librenms.org/t/errors-validate-php/24970/3
@Pelt10 I just switched snmpsim to use a venv in #15471 If you could change your code to be in line with that, it would be great. For example, it stores the venv in .python_venvs
.python_venvs
Why the s at the end?
@slalomsk8er because I assume we would want a different one for snmpsim vs the dispatcher.
@murrant ah, the s stands for snmpsim - I would have chosen something like .python_venv_s
or .python_venv_snmpsim
.
Also why is it hidden, git?
BTW the steps from https://github.com/librenms/librenms/issues/15051#issuecomment-1855341079 & https://github.com/librenms/librenms/issues/15051#issuecomment-1857965364 but with .python_venvs
worked well to upgrade my test system from Debian 11 to 12.
.python_venvs/snmpsim = snmpsim .python_venvs/dispatcher = dispatcher
The problem
Validation fails after a fresh installed on Debian 12
FAIL: Python3 module issue found: 'Required packages: ['PyMySQL!=1.0.0', 'python-dotenv', 'redis>=4.0', 'setuptools', 'psutil>=5.6.0', 'command_runner>=1.3.0'] Package not found: The 'command_runner>=1.3.0' distribution was not found and is required by the application '
Most but not all packages are installed in an externally-managed-environment
Output of ./validate.php
What was the last working version of LibreNMS?
No response
Anything in the logs that might be useful for us?