Closed m10k closed 5 months ago
I have been able to set up a working environment with the following commands.
$ alma-sbom/
$ python3 -m venv hoge
$ . hoge/bin/activate
$ python3 setup.py install
$ pip3 install google-api-core==2.14.0 google-auth==2.23.4 googleapis-common-protos==1.61.0 grpcio==1.60.0rc1
$ pip3 install --upgrade protobuf
The environment has the following modules installed.
alma-sbom==0.0.2
appier==1.31.4
beartype==0.16.4
boolean.py==4.0
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.11
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.14.0
google-auth==2.23.4
googleapis-common-protos==1.61.0
grpcio==1.60.0rc1
idna==3.6
immudb-py==1.4.0
immudb-wrapper==0.1.1
isodate==0.6.1
license-expression==30.2.0
packageurl-python==0.10.3
ply==3.11
protobuf==4.25.1
pyasn1==0.5.1
pyasn1-modules==0.3.0
pyparsing==3.1.1
PyYAML==6.0.1
rdflib==7.0.0
requests==2.31.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
uritools==4.0.2
urllib3==1.26.18
xmltodict==0.13.0
The difference between this one and the previous (broken) environment is the following:
-google-api-core==2.15.0
+google-api-core==2.14.0
-google-auth==2.25.1
+google-auth==2.23.4
-googleapis-common-protos==1.62.0
+googleapis-common-protos==1.61.0
-grpcio==1.60.0
+grpcio==1.60.0rc1
-protobuf==3.20.3
+protobuf==4.25.1
Following above report, i also investigated this issue. On my environment, i needed only two change about protobuf-related packages
$ cd alma-sbom/
$ python3.9 -m venv env
$ source env/bin/activate
(env) $ python setup.py install
(env) $ pip install googleapis-common-protos==1.61.0
(env) $ pip install --upgrade protobuf
Downgrading google-api-core and google-auth may not be needed.
Eventually, pip freeze
result is this.
(env) $ pip freeze
alma-sbom==0.0.2
appier==1.31.4
beartype==0.16.4
boolean.py==4.0
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.11
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.15.0
google-auth==2.25.1
googleapis-common-protos==1.61.0
grpcio==1.60.0
idna==3.6
immudb-py==1.4.0
immudb-wrapper==0.1.1
isodate==0.6.1
license-expression==30.2.0
packageurl-python==0.10.3
ply==3.11
protobuf==4.25.1
pyasn1==0.5.1
pyasn1-modules==0.3.0
pyparsing==3.1.1
PyYAML==6.0.1
rdflib==7.0.0
requests==2.31.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
uritools==4.0.2
urllib3==1.26.18
xmltodict==0.13.0
I noticed a problem in the workaround on my above comment (https://github.com/AlmaLinux/build-system/issues/232).
(env) $ pip install googleapis-common-protos==1.61.0
(env) $ pip install --upgrade protobuf
When upgrade protobuf, then we can get latest version (now protobuf==4.25.3). But immudb-py has dependency 'protobuf>=3.13.0,<4.0.0' https://github.com/codenotary/immudb-py/blob/master/setup.py#L34
Thus, upgrading protobuf breaks the dependency.
Upon further investigation of this issue, I have found the following.
Following these steps, you will not encounter this error:
pip install
. (Instead of immudb-py, immudb-wrapper is also OK)python setup.py install
.
(testenv01) $ python -m pip install immudb-py
(testenv01) $ python setup.py install
(testenv01) $ python alma_sbom.py --help
usage: alma_sbom.py [-h] [--output-file OUTPUT_FILE] [--file-format [{"cyclonedx-json","cyclonedx-xml","spdx-json","spdx-tagvalue","spdx-xml","spdx-yaml"}]]
(--build-id BUILD_ID | --rpm-package-hash RPM_PACKAGE_HASH) [--albs-url ALBS_URL] [--immudb-username IMMUDB_USERNAME]
[--immudb-password IMMUDB_PASSWORD] [--immudb-database IMMUDB_DATABASE] [--immudb-address IMMUDB_ADDRESS]
[--immudb-public-key-file IMMUDB_PUBLIC_KEY_FILE] [--verbose] [--debug]
optional arguments: -h, --help show this help message and exit (...)
Furthermore, I encounterd the following bizarre phenomena.
After running setup.py and encountering this error, I uninstalled googleapis-common-protos and protobuf and reinstalled these liblaries with pip install.
The result of pip freeze was the same, but the error not occured
(testenv02) $ python setup.py install
(testenv02) $ python alma_sbom.py --help
Traceback (most recent call last):
File "/home/khwarizmi/git/alma-sbom/alma_sbom.py", line 13, in
(testenv02) $ python -m pip uninstall googleapis-common-protos Successfully uninstalled googleapis-common-protos-1.62.0 (testenv02) $ python -m pip install googleapis-common-protos==1.62.0 Successfully installed googleapis-common-protos-1.62.0 (testenv02) $ python -m pip uninstall protobuf Successfully uninstalled protobuf-3.20.3 (testenv02) $ python -m pip install protobuf==3.20.3 Successfully installed protobuf-3.20.3
$ python alma_sbom.py --help usage: alma_sbom.py [-h] [--output-file OUTPUT_FILE] [--file-format [{"cyclonedx-json","cyclonedx-xml","spdx-json","spdx-tagvalue","spdx-xml","spdx-yaml"}]] (--build-id BUILD_ID | --rpm-package-hash RPM_PACKAGE_HASH) [--albs-url ALBS_URL] [--immudb-username IMMUDB_USERNAME] [--immudb-password IMMUDB_PASSWORD] [--immudb-database IMMUDB_DATABASE] [--immudb-address IMMUDB_ADDRESS] [--immudb-public-key-file IMMUDB_PUBLIC_KEY_FILE] [--verbose] [--debug]
optional arguments: -h, --help show this help message and exit4 (...)
These results indicate that (i guess):
- If googleapis-common-protos and protobuf are installed by `pip install`, it worked.
- If googleapis-common-protos and protobuf are installed through `setup.py`. it doesn't worked.
While alma-sbom.py is used here, the phenomenon can also be reproduced with a simple script that only imports immudb modules:
from immudb import ImmudbClient
The issue could be due to a problem with immudb-py, or googleapis-common-protos and protobuf.
If anyone has any knowledge about this issue, please let me know.
(The above comment has become long, so I will list workaround again below.)
Current Correct Workaround:
### first, deactivate and remove your venv
$ python3.9 -m venv env
$ source env/bin/activate
(env) $ python -m pip install immudb-py
(env) $ python setup.py install
(env) $ python alma_sbom.py --help
Thanks for the detailed explanation.
As far as I see, there is a problem with the googleapis-common-protos
egg being installed by setuptools, which is not generating the google
namespace properly.
I believe that a temporary solution could be to add both googleapis-common-protos==1.61.0
and protobuf==3.20.3
as deps in immudb-wrapper and things should work again. In any case, looks like this needs to be reported in python-api-common-protos, as pinning to 1.61.0
solves the issue.
@KAWAHARA-souta can you check if https://github.com/AlmaLinux/immudb-wrapper/pull/8 fixes the issue for you?
Thank you for your work @javihernandez !! It looks OK for me.
When executing alma-sbom on a clean AlmaLinux 9 machine, I'm getting the following error (after running
python3 setup.py install
).According to
pip3 freeze
, the following dependencies are installed:I have another (older) environment, which is still working. This environment has the following packages installed:
I tried downgrading some of the google packages, and while that fixed the problem with
google.api
, it caused a similar error forgoogle.protobuf
. Both environments have the same alma-sbom and immudb-wrapper installed, so something in the dependencies must have changed.