No module named 'opengrok_tools' in Docker

[vladak]

The latest Docker images cannot be started due to:

opengrok | Traceback (most recent call last):
opengrok |   File "/scripts/start.py", line 41, in <module>
opengrok |     from opengrok_tools.config_merge import merge_config_files
opengrok | ModuleNotFoundError: No module named 'opengrok_tools'
opengrok exited with code 1

[vladak]

when building the image, the step than runs python3 -m pip install --no-cache-dir /opengrok/tools/opengrok-tools* produces this output:

Processing /opengrok/tools/opengrok-tools.tar.gz
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Installing backend dependencies: started
  Installing backend dependencies: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Building wheels for collected packages: UNKNOWN
  Building wheel for UNKNOWN (pyproject.toml): started
  Building wheel for UNKNOWN (pyproject.toml): finished with status 'done'
  Created wheel for UNKNOWN: filename=UNKNOWN-0.0.0-py3-none-any.whl size=7352 sha256=a12c5e1c3ab3c57bf9fedf5cd039917f369041dbab523d7908679923f5b0cef8
  Stored in directory: /tmp/pip-ephem-wheel-cache-8pqb7050/wheels/dc/83/33/873e716fc69ebf961f2dd44d2cd681108d85a094b11cb179d6
Successfully built UNKNOWN
Installing collected packages: UNKNOWN
Successfully installed UNKNOWN-0.0.0

normally (in non Docker environment) the install output looks like this:

Processing ./opengrok-tools.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
  Preparing metadata (pyproject.toml) ... done
Collecting requests>=2.20.0
  Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Collecting filelock
  Using cached filelock-3.13.1-py3-none-any.whl (11 kB)
Collecting pyyaml
  Using cached PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (705 kB)
Collecting resource
  Using cached Resource-0.2.1-py2.py3-none-any.whl (25 kB)
Collecting jsonschema==2.6.0
  Using cached jsonschema-2.6.0-py2.py3-none-any.whl (39 kB)
Collecting charset-normalizer<4,>=2
  Using cached charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (142 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2023.11.17-py3-none-any.whl (162 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.6-py3-none-any.whl (61 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 61.6/61.6 KB 1.4 MB/s eta 0:00:00
Collecting urllib3<3,>=1.21.1
  Downloading urllib3-2.1.0-py3-none-any.whl (104 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 104.6/104.6 KB 6.8 MB/s eta 0:00:00
Collecting JsonForm>=0.0.2
  Using cached JsonForm-0.0.2-py3-none-any.whl
Collecting python-easyconfig>=0.1.0
  Using cached Python_EasyConfig-0.1.7-py2.py3-none-any.whl (5.4 kB)
Collecting JsonSir>=0.0.2
  Using cached JsonSir-0.0.2-py3-none-any.whl
Collecting six
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Building wheels for collected packages: opengrok-tools
  Building wheel for opengrok-tools (pyproject.toml) ... done
  Created wheel for opengrok-tools: filename=opengrok_tools-1.12.25-py3-none-any.whl size=77596 sha256=d2ec50050b77269d7a901a040ce7c5ef9143bb95128bcabcbe0811663063aec4
  Stored in directory: /home/vkotal/.cache/pip/wheels/76/6b/58/1d9871c518d2af881e96f9a4d6ad4502c009ed638716f36849
Successfully built opengrok-tools
Installing collected packages: JsonSir, jsonschema, urllib3, six, pyyaml, JsonForm, idna, filelock, charset-normalizer, certifi, requests, python-easyconfig, resource, opengrok-tools
Successfully installed JsonForm-0.0.2 JsonSir-0.0.2 certifi-2023.11.17 charset-normalizer-3.3.2 filelock-3.13.1 idna-3.6 jsonschema-2.6.0 opengrok-tools-1.12.25 python-easyconfig-0.1.7 pyyaml-6.0.1 requests-2.31.0 resource-0.2.1 six-1.16.0 urllib3-2.1.0

[cvasani]

is there any workaround to get this working ? I am also trying to build image for linux/arm64 and not able to succeed. If any one already have a linux/arm64 docker image that I can pull and use it for some time that would help. Thanks

[vladak]

is there any workaround to get this working ? I am also trying to build image for linux/arm64 and not able to succeed. If any one already have a linux/arm64 docker image that I can pull and use it for some time that would help. Thanks

no workaround that I know of.

[vladak]

I did a small experiment to reproduce this: docker run tomcat:10.1.13-jdk17, docker exec -it ... bash and inside the container I installed python3, python3-pip, python3-venv, python3-wheel, downloaded the latest opengrok-tools.tar.gz and installed it with python3 -m pip install --no-cache-dir opengrok-tools.tar.gz - exactly like it is done in the Dockerfile. This reproduced the problem - the package was not detected and "installed" as UNKNOWN. When I created Python venv inside that container and installed the package there, it was installed without a problem and I could run opengrok-sync.

My current hypothesis is that the something in the Python 3.10.12 ecosystem inside the image is either too old or there is something missing.

System-wide pip and venv based pip are of the same version: 22.0.2, the python3-setuptools APT pakage is at 59.6.0-1.2ubuntu0.22.04.1.

Found 2 related issues in different project https://github.com/karlotness/adrt/issues/1 and in setuptools: https://github.com/pypa/setuptools/discussions/3691

[vladak]

Tried the workaround from https://stackoverflow.com/a/53081068/11582827 , i.e. create setup.py, and that lead to successful detection of the opengrok-tools package, however it was still not installed properly.

Ugrading pip via python3 -m pip install --upgrade pip to 23.3.1 and installing the original package (without setup.py) lead to successful installation.