No module named 'google.api'

[m10k]

When executing alma-sbom on a clean AlmaLinux 9 machine, I'm getting the following error (after running python3 setup.py install).

(env) [root@alma-sbom-env alma-sbom]# ./alma_sbom.py --help
Traceback (most recent call last):
  File "/root/alma-sbom/./alma_sbom.py", line 13, in <module>
    from immudb_wrapper import ImmudbWrapper
  File "<frozen zipimport>", line 259, in load_module
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_wrapper-0.1.1-py3.9.egg/immudb_wrapper.py", line 15, in <module>
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/__init__.py", line 13, in <module>
    from immudb.client import ImmudbClient
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/client.py", line 18, in <module>
    from immudb import grpcutils
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpcutils.py", line 20, in <module>
    import immudb.grpc.schema_pb2_grpc as schema_pb2_grpc
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpc/schema_pb2_grpc.py", line 6, in <module>
    from . import schema_pb2 as schema__pb2
  File "/root/alma-sbom/env/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpc/schema_pb2.py", line 16, in <module>
    from google.api import annotations_pb2 as google_dot_api_dot_annotations__pb2
ModuleNotFoundError: No module named 'google.api'

According to pip3 freeze, the following dependencies are installed:

alma-sbom==0.0.2
appier==1.31.4
beartype==0.16.4
boolean.py==4.0
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.11
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.15.0
google-auth==2.25.1
googleapis-common-protos==1.62.0
grpcio==1.60.0
idna==3.6
immudb-py==1.4.0
immudb-wrapper==0.1.1
isodate==0.6.1
license-expression==30.2.0
packageurl-python==0.10.3
ply==3.11
protobuf==3.20.3
pyasn1==0.5.1
pyasn1-modules==0.3.0
pyparsing==3.1.1
PyYAML==6.0.1
rdflib==7.0.0
requests==2.31.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
uritools==4.0.2
urllib3==1.26.18
xmltodict==0.13.0

I have another (older) environment, which is still working. This environment has the following packages installed:

alma-sbom==0.0.1
appier==1.31.4
beartype==0.15.0
boolean.py==4.0
cachetools==5.3.2
cas-wrapper==0.0.6
certifi==2023.5.7
charset-normalizer==3.1.0
click==8.1.3
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.10
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.14.0
google-auth==2.23.4
googleapis-common-protos==1.61.0
grpcio==1.60.0rc1
idna==3.4
immudb-py==1.4.0
immudb-wrapper==0.1.2
isodate==0.6.1
license-expression==30.1.1
packageurl-python==0.10.3
plumbum==1.8.1
ply==3.11
protobuf==3.20.3
pyasn1==0.5.0
pyasn1-modules==0.3.0
pyparsing==3.0.9
PyYAML==6.0
rdflib==6.3.2
requests==2.30.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.0
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
UNKNOWN==0.0.0
uritools==4.0.1
urllib3==2.0.2
xmltodict==0.13.0

I tried downgrading some of the google packages, and while that fixed the problem with google.api, it caused a similar error for google.protobuf. Both environments have the same alma-sbom and immudb-wrapper installed, so something in the dependencies must have changed.

[m10k]

I have been able to set up a working environment with the following commands.

$ alma-sbom/
$ python3 -m venv hoge
$ . hoge/bin/activate
$ python3 setup.py install
$ pip3 install google-api-core==2.14.0 google-auth==2.23.4 googleapis-common-protos==1.61.0 grpcio==1.60.0rc1
$ pip3 install --upgrade protobuf

The environment has the following modules installed.

alma-sbom==0.0.2
appier==1.31.4
beartype==0.16.4
boolean.py==4.0
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.11
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.14.0
google-auth==2.23.4
googleapis-common-protos==1.61.0
grpcio==1.60.0rc1
idna==3.6
immudb-py==1.4.0
immudb-wrapper==0.1.1
isodate==0.6.1
license-expression==30.2.0
packageurl-python==0.10.3
ply==3.11
protobuf==4.25.1
pyasn1==0.5.1
pyasn1-modules==0.3.0
pyparsing==3.1.1
PyYAML==6.0.1
rdflib==7.0.0
requests==2.31.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
uritools==4.0.2
urllib3==1.26.18
xmltodict==0.13.0

The difference between this one and the previous (broken) environment is the following:

-google-api-core==2.15.0
+google-api-core==2.14.0
-google-auth==2.25.1
+google-auth==2.23.4
-googleapis-common-protos==1.62.0
+googleapis-common-protos==1.61.0
-grpcio==1.60.0
+grpcio==1.60.0rc1
-protobuf==3.20.3
+protobuf==4.25.1

[KAWAHARA-souta]

Following above report, i also investigated this issue. On my environment, i needed only two change about protobuf-related packages

$ cd alma-sbom/
$ python3.9 -m venv env
$ source env/bin/activate
(env) $ python setup.py install
(env) $ pip install googleapis-common-protos==1.61.0
(env) $ pip install --upgrade protobuf

Downgrading google-api-core and google-auth may not be needed. Eventually, pip freeze result is this.

(env) $ pip freeze
alma-sbom==0.0.2
appier==1.31.4
beartype==0.16.4
boolean.py==4.0
cachetools==5.3.2
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
cyclonedx-python-lib==2.7.1
dataclasses==0.8
ecdsa==0.18.0
gitdb==4.0.11
GitPython==3.1.29
google-api==0.1.12
google-api-core==2.15.0
google-auth==2.25.1
googleapis-common-protos==1.61.0
grpcio==1.60.0
idna==3.6
immudb-py==1.4.0
immudb-wrapper==0.1.1
isodate==0.6.1
license-expression==30.2.0
packageurl-python==0.10.3
ply==3.11
protobuf==4.25.1
pyasn1==0.5.1
pyasn1-modules==0.3.0
pyparsing==3.1.1
PyYAML==6.0.1
rdflib==7.0.0
requests==2.31.0
rsa==4.9
semantic-version==2.10.0
six==1.16.0
smmap==5.0.1
sortedcontainers==2.4.0
spdx-tools==0.8.0
toml==0.10.2
uritools==4.0.2
urllib3==1.26.18
xmltodict==0.13.0

[KAWAHARA-souta]

I noticed a problem in the workaround on my above comment (https://github.com/AlmaLinux/build-system/issues/232).

(env) $ pip install googleapis-common-protos==1.61.0
(env) $ pip install --upgrade protobuf

When upgrade protobuf, then we can get latest version (now protobuf==4.25.3). But immudb-py has dependency 'protobuf>=3.13.0,<4.0.0' https://github.com/codenotary/immudb-py/blob/master/setup.py#L34

Thus, upgrading protobuf breaks the dependency.

Upon further investigation of this issue, I have found the following.

Following these steps, you will not encounter this error:

• First, install immudb-py by pip install . (Instead of immudb-py, immudb-wrapper is also OK)
• And then run python setup.py install.

  
  (testenv01) $ python -m pip install immudb-py
  (testenv01) $ python setup.py install
  (testenv01) $ python alma_sbom.py --help
  usage: alma_sbom.py [-h] [--output-file OUTPUT_FILE] [--file-format [{"cyclonedx-json","cyclonedx-xml","spdx-json","spdx-tagvalue","spdx-xml","spdx-yaml"}]]
                  (--build-id BUILD_ID | --rpm-package-hash RPM_PACKAGE_HASH) [--albs-url ALBS_URL] [--immudb-username IMMUDB_USERNAME]
                  [--immudb-password IMMUDB_PASSWORD] [--immudb-database IMMUDB_DATABASE] [--immudb-address IMMUDB_ADDRESS]
                  [--immudb-public-key-file IMMUDB_PUBLIC_KEY_FILE] [--verbose] [--debug]

optional arguments: -h, --help show this help message and exit (...)

Furthermore, I encounterd the following bizarre phenomena.
After running setup.py and encountering this error, I uninstalled googleapis-common-protos and protobuf and reinstalled these liblaries with pip install. 
The result of pip freeze was the same, but the error not occured

(testenv02) $ python setup.py install (testenv02) $ python alma_sbom.py --help Traceback (most recent call last): File "/home/khwarizmi/git/alma-sbom/alma_sbom.py", line 13, in from immudb_wrapper import ImmudbWrapper File "", line 259, in load_module File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_wrapper-0.1.1-py3.9.egg/immudb_wrapper.py", line 15, in File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/init.py", line 13, in from immudb.client import ImmudbClient File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/client.py", line 18, in from immudb import grpcutils File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpcutils.py", line 20, in import immudb.grpc.schema_pb2_grpc as schema_pb2_grpc File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpc/schema_pb2_grpc.py", line 6, in from . import schema_pb2 as schema__pb2 File "/home/khwarizmi/git/alma-sbom/testenv02/lib64/python3.9/site-packages/immudb_py-1.4.0-py3.9.egg/immudb/grpc/schema_pb2.py", line 16, in from google.api import annotations_pb2 as google_dot_api_dot_annotations__pb2 ModuleNotFoundError: No module named 'google.api'

(testenv02) $ python -m pip uninstall googleapis-common-protos Successfully uninstalled googleapis-common-protos-1.62.0 (testenv02) $ python -m pip install googleapis-common-protos==1.62.0 Successfully installed googleapis-common-protos-1.62.0 (testenv02) $ python -m pip uninstall protobuf Successfully uninstalled protobuf-3.20.3 (testenv02) $ python -m pip install protobuf==3.20.3 Successfully installed protobuf-3.20.3

$ python alma_sbom.py --help usage: alma_sbom.py [-h] [--output-file OUTPUT_FILE] [--file-format [{"cyclonedx-json","cyclonedx-xml","spdx-json","spdx-tagvalue","spdx-xml","spdx-yaml"}]] (--build-id BUILD_ID | --rpm-package-hash RPM_PACKAGE_HASH) [--albs-url ALBS_URL] [--immudb-username IMMUDB_USERNAME] [--immudb-password IMMUDB_PASSWORD] [--immudb-database IMMUDB_DATABASE] [--immudb-address IMMUDB_ADDRESS] [--immudb-public-key-file IMMUDB_PUBLIC_KEY_FILE] [--verbose] [--debug]

optional arguments: -h, --help show this help message and exit4 (...)

These results indicate that (i guess):
 - If googleapis-common-protos and protobuf are installed by `pip install`, it worked.
 - If googleapis-common-protos and protobuf are installed through `setup.py`. it doesn't worked. 

While alma-sbom.py is used here, the phenomenon can also be reproduced with a simple script that only imports immudb modules:

from immudb import ImmudbClient

The issue could be due to a problem with immudb-py, or  googleapis-common-protos and protobuf.

If anyone has any knowledge about this issue, please let me know.

[KAWAHARA-souta]

(The above comment has become long, so I will list workaround again below.)

Current Correct Workaround:

### first, deactivate and remove your venv
$ python3.9 -m venv env
$ source env/bin/activate
(env) $ python -m pip install immudb-py
(env) $ python setup.py install
(env) $ python alma_sbom.py --help

[javihernandez]

Thanks for the detailed explanation. As far as I see, there is a problem with the googleapis-common-protos egg being installed by setuptools, which is not generating the google namespace properly. I believe that a temporary solution could be to add both googleapis-common-protos==1.61.0 and protobuf==3.20.3 as deps in immudb-wrapper and things should work again. In any case, looks like this needs to be reported in python-api-common-protos, as pinning to 1.61.0 solves the issue.

[javihernandez]

@KAWAHARA-souta can you check if https://github.com/AlmaLinux/immudb-wrapper/pull/8 fixes the issue for you?

[KAWAHARA-souta]

Thank you for your work @javihernandez !! It looks OK for me.