SHT31 dependencies not recognized after install

[emuehlstein]

Adding an SHT31 sensor to a fresh mycodo install. The dependency install returns "All Dependencies have been installed"; however, when I attempt to add the sensor again, it says the "Adafruit-SHT31" dependency is unmet and must be installed and the process repeats.

Versions:

• Mycodo Version: 8.6.4
• Raspberry Pi Version: 3B
• Raspbian OS Version: Buster

Reproducibility

Please list specific setup details that are involved and the steps to reproduce the behavior:

1. Add an SHT31 sensor
2. Install dependencies
3. Add an SHT31 sensor
4. Goto #2

Expected behavior

Sensor dependencies installed correctly, second attempt at adding sensor successful.

Additional context

Dependency install output:

[2020-07-28 00:41:43] Installing collected packages: adafruit-sht31 [2020-07-28 00:41:43] Attempting uninstall: adafruit-sht31 [2020-07-28 00:41:43] Found existing installation: Adafruit-SHT31 1.0.2 [2020-07-28 00:41:43] Uninstalling Adafruit-SHT31-1.0.2: [2020-07-28 00:41:43] Successfully uninstalled Adafruit-SHT31-1.0.2 [2020-07-28 00:41:43] Running setup.py develop for adafruit-sht31 [2020-07-28 00:41:46] Successfully installed adafruit-sht31

[2020-07-28 00:41:46] End install of Adafruit_SHT31

[2020-07-28 00:41:47] [2020-07-28 00:41:47] #### Setting permissions

[2020-07-28 00:41:47] #### All Dependencies have been installed.

[2020-07-28 00:50:33] Dependency installation beginning. Installing: Adafruit_SHT31

[2020-07-28 00:50:33] [2020-07-28 00:50:33] #### Installing/updating git://github.com/ralf1070/Adafruit_Python_SHT31.git#egg=adafruit-sht31 (pip-git) [2020-07-28 00:50:36] Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple [2020-07-28 00:50:36] Obtaining adafruit-sht31 from git+git://github.com/ralf1070/Adafruit_Python_SHT31.git#egg=adafruit-sht31 [2020-07-28 00:50:36] Updating ./env/src/adafruit-sht31 clone [2020-07-28 00:50:36] Running command git fetch -q --tags [2020-07-28 00:50:36] Running command git reset --hard -q 9bd7c4131f0d8f4132ceeac15faf2b92180b953f [2020-07-28 00:50:41] Installing collected packages: adafruit-sht31 [2020-07-28 00:50:41] Attempting uninstall: adafruit-sht31 [2020-07-28 00:50:41] Found existing installation: Adafruit-SHT31 1.0.2 [2020-07-28 00:50:41] Uninstalling Adafruit-SHT31-1.0.2: [2020-07-28 00:50:41] Successfully uninstalled Adafruit-SHT31-1.0.2 [2020-07-28 00:50:41] Running setup.py develop for adafruit-sht31 [2020-07-28 00:50:44] Successfully installed adafruit-sht31

[2020-07-28 00:50:45] End install of Adafruit_SHT31

[2020-07-28 00:50:45] [2020-07-28 00:50:45] #### Setting permissions

[2020-07-28 00:50:46] #### All Dependencies have been installed.

[kizniche]

I was able to reproduce the issue. It's strange, because the module successfully installs. I'll investigate.

[kizniche]

This is the code of interest that's failing:

import importlib
module = importlib.util.find_spec('Adafruit_SHT31')
print(module)

Executed in the Mycodo Python environment, it works, yielding:

pi@hydra:~/Mycodo $ ./env/bin/python ./test_mod_install_01.py
ModuleSpec(name='Adafruit_SHT31', loader=<_frozen_importlib_external.SourceFileLoader object at 0xb665d110>, origin='/home/pi/Mycodo/env/src/adafruit-sht31/Adafruit_SHT31.py')

However, in Mycodo, the same code fails, yielding None:

https://github.com/kizniche/Mycodo/blob/b737eca252cc978663c000b9a484afa1e0c007e8/mycodo/mycodo_flask/utils/utils_general.py#L1126-L1131

Very strange. I'm not sure where to go from here.

[kizniche]

This is strange. I ran the following code from the terminal, within the virtualenv, and then from within Mycodo (Flask) in the virtualenv:

import pkg_resources
installed = [pkg.key for pkg in pkg_resources.working_set]
installed.sort()
print(installed)

Terminal output:

['adafruit-ads1x15', 'adafruit-blinka', 'adafruit-circuitpython-ads1x15', 'adafruit-circuitpython-adt7410', 'adafruit-circuitpython-adxl34x', 'adafruit-circuitpython-busdevice', 'adafruit-circuitpython-register', 'adafruit-extended-bus', 'adafruit-gpio', 'adafruit-platformdetect', 'adafruit-pureio', 'adafruit-sht31', 'adafruit-ssd1306', 'alembic', 'aniso8601', 'appdirs', 'argparse', 'astroid', 'attrs', 'babel', 'bcrypt', 'beautifulsoup4', 'brotli', 'certifi', 'cffi', 'chardet', 'click', 'daemonize', 'decorator', 'distlib', 'distro', 'dnspython', 'email-validator', 'factory-boy', 'faker', 'filelock', 'flask', 'flask-accept', 'flask-babel', 'flask-compress', 'flask-limiter', 'flask-login', 'flask-marshmallow', 'flask-restx', 'flask-sqlalchemy', 'flask-sslify', 'flask-talisman', 'flask-wtf', 'future', 'geocoder', 'gunicorn', 'idna', 'importlib-metadata', 'influxdb', 'iso8601', 'isort', 'itsdangerous', 'jinja2', 'jsonschema', 'lazy-object-proxy', 'limits', 'logilab-common', 'mako', 'markupsafe', 'marshmallow', 'marshmallow-sqlalchemy', 'mccabe', 'mock', 'more-itertools', 'msgpack', 'packaging', 'paho-mqtt', 'picamera', 'pigpio', 'pillow', 'pip', 'pluggy', 'psutil', 'ptyprocess', 'py', 'pycparser', 'pyftdi', 'pylibftdi', 'pylint', 'pylint-common', 'pylint-plugin-utils', 'pyparsing', 'pyro5', 'pyrsistent', 'pyserial', 'pytest', 'python-apt', 'python-dateutil', 'python-editor', 'pytz', 'pyusb', 'pyyaml', 'ratelim', 'requests', 'rpi-ws281x', 'rpi.gpio', 'ruuvitag-sensor', 'rx', 'seeed-grove.py', 'seeed-python-ds18b20', 'serpent', 'setuptools', 'six', 'smbus2', 'soupsieve', 'spidev', 'sqlalchemy', 'ssh-import-id', 'sysv-ipc', 'testfixtures', 'text-unidecode', 'urllib3', 'virtualenv', 'w1thermsensor', 'waitress', 'wcwidth', 'webob', 'webtest', 'werkzeug', 'wheel', 'wrapt', 'wtforms', 'zipp']

Mycodo output:

['adafruit-ads1x15', 'adafruit-blinka', 'adafruit-circuitpython-adt7410', 'adafruit-circuitpython-adxl34x', 'adafruit-circuitpython-busdevice', 'adafruit-circuitpython-register', 'adafruit-extended-bus', 'adafruit-gpio', 'adafruit-platformdetect', 'adafruit-pureio', 'adafruit-ssd1306', 'alembic', 'aniso8601', 'appdirs', 'argparse', 'astroid', 'attrs', 'babel', 'bcrypt', 'beautifulsoup4', 'brotli', 'certifi', 'cffi', 'chardet', 'click', 'daemonize', 'decorator', 'distlib', 'distro', 'dnspython', 'email-validator', 'factory-boy', 'faker', 'filelock', 'flask', 'flask-accept', 'flask-babel', 'flask-compress', 'flask-limiter', 'flask-login', 'flask-marshmallow', 'flask-restx', 'flask-sqlalchemy', 'flask-sslify', 'flask-talisman', 'flask-wtf', 'future', 'geocoder', 'gunicorn', 'idna', 'importlib-metadata', 'influxdb', 'isort', 'itsdangerous', 'jinja2', 'jsonschema', 'lazy-object-proxy', 'limits', 'logilab-common', 'mako', 'markupsafe', 'marshmallow', 'marshmallow-sqlalchemy', 'mccabe', 'mock', 'more-itertools', 'msgpack', 'packaging', 'picamera', 'pigpio', 'pillow', 'pip', 'pluggy', 'psutil', 'ptyprocess', 'py', 'pycparser', 'pyftdi', 'pylibftdi', 'pylint', 'pylint-common', 'pylint-plugin-utils', 'pyparsing', 'pyro5', 'pyrsistent', 'pyserial', 'pytest', 'python-apt', 'python-dateutil', 'python-editor', 'pytz', 'pyusb', 'ratelim', 'requests', 'rpi-ws281x', 'rpi.gpio', 'ruuvitag-sensor', 'rx', 'seeed-grove.py', 'seeed-python-ds18b20', 'serpent', 'setuptools', 'six', 'smbus2', 'soupsieve', 'spidev', 'sqlalchemy', 'ssh-import-id', 'sysv-ipc', 'testfixtures', 'text-unidecode', 'urllib3', 'virtualenv', 'w1thermsensor', 'waitress', 'wcwidth', 'webob', 'webtest', 'werkzeug', 'wheel', 'wrapt', 'wtforms', 'zipp']

There are more modules listed when executed from the terminal than when executed from Mycodo. Here's the difference between the lists:

['adafruit-circuitpython-ads1x15', 'adafruit-sht31', 'iso8601', 'paho-mqtt', 'pyyaml']

So, I've only verified again that the adafruit-sht31 module is not being detected in Mycodo.

[emuehlstein]

Oddly, when I did a manual "pip install adafruit-sht31" on the command line the dependency was detected and I was able to configure the sensor.

[kizniche]

The virtualenv was created with system site packages enabled, so if you have a Python module installed outside the virtualenv (like you just did), modules will be first searched for outside the virtualenv before searching within the virtualenv. This isn't a solution, it merely masks the issue.

[kizniche]

Alright, I found the issue. You gave me an idea. Mycodo is installing the virtaulenv package with:

pip install -e git://github.com/ralf1070/Adafruit_Python_SHT31.git#egg=adafruit-sht31

I uninstalled adafruit-sht31 from the virtualenv and installed it within the virtualenv with:

pip install adafruit-sht31

Reloaded the Mycodo frontend, and it worked. The module is now detected in Mycodo in the virtualenv. This is strange. I need to read some more documentation to figure this out.

[kizniche]

Alright, I'm testing to see if this behavior exists with other modules that use this install format, and it does. This must be a change that occurred recently with pip or a package it relies on. Not good.

[kizniche]

Changing the format from:

pip install -e git://github.com/ralf1070/Adafruit_Python_SHT31.git#egg=adafruit-sht31

to

pip install git+https://github.com/ralf1070/Adafruit_Python_SHT31.git

Appears to fix the issue.

[kizniche]

I pushed a fix last night for all modules using this install format, which will be available in the next release.