ModuleNotFoundError: No module named 'sqlparse'

[ddmo]

Hi, after installation and configuration I got this error in zbxdb_starter.log:

Wed 14 Oct 2020 12:57:01 PM CEST Starting etc/zbxdb.odb.cfg
Traceback (most recent call last):
  File "/home/zbxdb/zbxdb/bin/zbxdb.py", line 36, in <module>
    import sqlparse
ModuleNotFoundError: No module named 'sqlparse'

Pyton is properly configured (I suppose):

$ type python
python is /home/zbxdb/.pyenv/shims/python

$ pip install sqlparse
Requirement already satisfied: sqlparse in /home/zbxdb/.pyenv/versions/3.6.5/lib/python3.6/site-packages

Whats wrong? Thanks a lot Stefano

[ddmo]

Ok, after several attempts I insert all vars in bash_profile and it works. Thanks

[ikzelf]

Did you enter the lines that are displayed at the end of the pyenv installation process in your .bashrc? Good that you were able to solve your issues, sorry that I did not see them earlier.

[wormhill]

Same problem here. Could you elaborate on what "inserting all vars" means? Did not work on pyenv's installation instructions, nor with copypasteing the .bashrc variables to .bash_profile.

.bash_profile:

export PYENV_ROOT="$HOME/.pyenv" export PATH="$PYENV_ROOT/bin:$PATH" eval "$(pyenv init --path)"

Get the aliases and functions

if [ -f ~/.bashrc ]; then . ~/.bashrc fi

User specific environment and startup programs

export ZBXDB_HOME=$HOME export ZBXDB_OUT=$ZBXDB_HOME/zbxdb_out; #make sure this reflects the out_dir parameter in the monitoring cfg files.) export PATH=$PATH:$HOME/zbxdb/bin

.bashrc:

Source global definitions

if [ -f /etc/bashrc ]; then . /etc/bashrc fi

User specific environment

if ! [[ "$PATH" =~ "$HOME/.local/bin:$HOME/bin:" ]] then PATH="$HOME/.local/bin:$HOME/bin:$PATH" fi

export PATH

User specific aliases and functions

export PATH="$HOME/.pyenv/bin:$PATH" eval "$(pyenv init -)" eval "$(pyenv virtualenv-init -)"

Am I missing something here?

[ikzelf]

Did you source the .bash_profile after adding the 3 lines for pyenv? (or did you logon again)

[wormhill]

Yeah, I did. Pip install sqlparse tells me inside the virtualenv that sqlparse is installed etc.

I'll try to reinstall the whole thing and let you know if it persists. Just in case something went awry.

[ikzelf]

How does the crontab entry for zbxdb_starter look like?

[wormhill]

Crontab seems fine since it runs the zbxdb_starter every minute. Copypasted straight from the manual.

The weird thing here is that running zbxdb.py -c etc/zbxdb.hostname.cfg by hand in the bin folder seems to work fine.

[ikzelf]

If it is running fine when interactive, check the crontab line for zbxdb_starter. It is important that it sources the .bash_profile before zbxdb_starter, otherwise the virtual environment function pyenv is not in the process environment.

[wormhill]

The crontab lines are as follows:

• • • • • . $HOME/.bash_profile;$HOME/zbxdb/bin/zbxdb_starter >log/zbxdb_starter.cron 2>&1
• • • • • . $HOME/.bash_profile;$HOME/zbxdb/bin/zbxdb_sender.py -c /etc/zabbix/zabbix_agentd.conf -z zbxdb_out >log/zbxdb_sender.cron 2>&1

Running on RHEL8.2.

[ikzelf]

Looks like OK. No complaints in log/zbxdb_starter.cron ?

[wormhill]

zbxdb_starter.cron:

mon etc/zbxdb.hostname.cfg 1

[ikzelf]

The zbxdb.py process for this config is still running. Stop it first and check the output of the first execution after stopping the zbxdb.py process. You can use zbxdb_stop to stop the running processes. If you change the log into append it gives you a bit more time to check the log: . $HOME/.bash_profile;$HOME/zbxdb/bin/zbxdb_starter >>log/zbxdb_starter.cron 2>&1

[wormhill]

Tried reinstalling the whole thing but the result is still the same. Tried zbxdb_stop, then add the append line to crontab and zbxdb_starter and after that it just appends the "mon etc/zbxdb.hostname.cfg 1" message every minute.

Zbxdb_starter.log only gives this:

Fri May 28 11:03:01 EEST 2021 Starting etc/zbxdb.hostname.cfg Traceback (most recent call last): File "/home/***/zbxdb/bin/zbxdb.py", line 39, in import sqlparse ModuleNotFoundError: No module named 'sqlparse'

When trying to import sqlparse from the virtual env like this:

cd zbxdb python

import sqlparse

It does not give any errors.

Pip list says that sqlparse is on version 0.4.1. Is that too new of a version?

[ikzelf]

I also have sqlparse 0.4.1 so that won't be the problem. python 3.9.2 ? I must confess that currently I have not the slightest idea what is wrong. If you followed https://github.com/ikzelf/zbxdb/blob/master/doc/getting_started.md, the setup part, it should work. Do you have a separate config file for pip in place? Your sqlparse shows up here: ls -ld ~/.pyenv/versions/3.9.2/envs/zbxdb-3.9.2/lib/python3.9/site-packages/sqlparse

[wormhill]

Yep, followed the setup part and configured all prereqs and pyenv variables etc. Sqlparse shows on that dir. No additional pip configs.

Should this work on a RHEL8/CentOS platform or should I try installing this on Ubuntu/Debian/something else? SELinux is on permissive mode btw.

[ikzelf]

I had it running on both OSX, RHEL and CentOS, with recent releases. I am not aware of any issues here. Can you do such an import test started from cron? Put a little script in zbxdb/z.py with just the import and see what happens. Normall when addressing a script in a directory that has a venv assigned to it, the venv is auto activated. For some reason it looks like this is not happening in your case.

[BernhardLinz]

try * * * * * . $HOME/.bash_profile;pyenv activate zbxdb-3.9.2;$HOME/zbxdb/bin/zbxdb_starter in crontab. I don not know why but it is always ignoring the pyenv settings

[ikzelf]

Maybe file this as an issue with the pyenv maintainer? Good tip btw.

[ikzelf]

Is /usr/local/bin in your PATH when coming from crontab? This is where pyenv is installed.

[ikzelf]

Did you get this working?

[BernhardLinz]

OS ist CentOS7, pyenv ist installed to .pyenv in the profile/home of the zbxdb user. For all users, pyenv is NOT installed. And yes, it is working with my posted crontab-solutions. I did not find out what is getting wrong on CentOS 7. I try difference settings in .bashrc and .bash_profile but all will be ignored.

[wormhill]

Installed a fresh OracleLinux 8.4 server and installed this there. Worked like a charm when using pyenv global zbxdb-3.9.2 instead of pyenv local zbxdb-3.9.2. I'll test that on the initial machine aswell.

What's the downside of using pyenv global? (Not a dev here. :))

[ikzelf]

The downside will be close to zero I guess (not a dev either ;-) ) but for me it looked cleaner to have this local. If your user is dedicated to running zbxdb the downside will even be closer to zero. The 'global' part is just for the user, not for the machine.