gnocchi 4.4.2 is incompatiable with openstack upper-constraits

[SeanMooney]

Before reporting an issue on Gnocchi, please be sure to provide all necessary information.

Which version of Gnocchi are you using

4.4.2 (current latest release on pypi)

How to reproduce your problem

install gnocchi 4.4.2 with pprotobuf===4.21.7

What is the result that you get

https://zuul.opendev.org/t/openstack/build/099e20697f57467bac6a25782e35645b/log/controller/logs/screen-gnocchi-api.txt#409 ` Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: 2023-01-31 17:48:00,494 [134704] CRITICAL root: Traceback (most recent call last): Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/bin/gnocchi-api", line 21, in Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: application = api.wsgi() Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/cli/api.py", line 53, in wsgi Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return app.load_app(prepare_service()) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/app.py", line 180, in load_app Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: app = deploy.loadapp("config:" + cfg_path, name=appname, Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 253, in loadapp Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return loadobj(APP, uri, name=name, kw) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 278, in loadobj Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return context.create() Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return self.object_type.invoke(self) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 148, in invoke Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return fix_call(context.object, Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 58, in fix_call Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: reraise(exc_info) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/compat.py", line 32, in reraise Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: raise e.with_traceback(tb) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 55, in fix_call Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: val = callable(args, kw) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/urlmap.py", line 31, in urlmap_factory Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: app = loader.get_app(app_name, global_conf=global_conf) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 355, in get_app Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return self.app_context( Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return self.object_type.invoke(self) Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 209, in invoke Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: app = context.app_context.create() Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 715, in create Jan 31 17:48:00.494490 np0032923073 devstack@gnocchi-api.service[134704]: return self.object_type.invoke(self) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/loadwsgi.py", line 152, in invoke Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: return fix_call(context.object, context.global_conf, context.local_conf) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 58, in fix_call Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: reraise(exc_info) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/compat.py", line 32, in reraise Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: raise e.with_traceback(tb) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/paste/deploy/util.py", line 55, in fix_call Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: val = callable(args, kw) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/app.py", line 203, in app_factory Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: return _setup_app(root=local_conf.get('root'), **appconfig) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/app.py", line 189, in _setup_app Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: hooks=(GnocchiHook(conf),), Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/app.py", line 55, in init Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: self.auth_helper = driver.DriverManager("gnocchi.rest.auth_helper", Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/driver.py", line 54, in init Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: super(DriverManager, self).init( Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/named.py", line 78, in init Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: extensions = self._load_plugins(invoke_on_load, Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/extension.py", line 241, in _load_plugins Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: self._on_load_failure_callback(self, ep, err) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/extension.py", line 229, in _load_plugins Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: ext = self._load_one_plugin(ep, Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/named.py", line 156, in _load_one_plugin Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: return super(NamedExtensionManager, self)._load_one_plugin( Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/stevedore/extension.py", line 263, in _load_one_plugin Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: plugin = ep.load() Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/lib64/python3.9/importlib/metadata.py", line 86, in load Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: module = import_module(match.group('module')) Jan 31 17:48:00.495176 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/lib64/python3.9/importlib/init.py", line 127, in import_module Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: return _bootstrap._gcd_import(name[level:], package, level) Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 1030, in _gcd_import Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 1007, in _find_and_load Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 986, in _find_and_load_unlocked Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 680, in _load_unlocked Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 850, in exec_module Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "", line 228, in _call_with_frames_removed Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/auth_helper.py", line 20, in Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: from gnocchi.rest import api Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/api.py", line 51, in Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: from gnocchi.rest.prometheus import remote_pb2 Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib/python3.9/site-packages/gnocchi/rest/prometheus/remote_pb2.py", line 33, in Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: _descriptor.EnumValueDescriptor( Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: File "/usr/local/lib64/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in new Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: _message.Message._CheckCalledFromGeneratedFile() Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: TypeError: Descriptors cannot not be created directly. Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: If you cannot immediately regenerate your protos, some other possible workarounds are: Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: 1. Downgrade the protobuf package to 3.20.x or lower. Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower). Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates Jan 31 17:48:00.495927 np0032923073 devstack@gnocchi-api.service[134704]: unable to load app 0 (mountpoint='') (callable not found or import error)

`

What is result that you expected

gnocci can be run with protobuf >3.20

this is cause by https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

and the issues have been fixed already in https://github.com/gnocchixyz/gnocchi/issues/1267 however there has not been a release since then. so while master works 4.4.2 does not.

can we do a 4.5.0 release to enable pypi packaged to be used instead of just master?

[SeanMooney]

@tobias-urdin is this something you can help with

[tobias-urdin]

I will look into this. I was hoping on getting SQLAlchemy 2.0 support and then release 5.0 but perhaps I should just cut a new stable/4.5 from master.

[SeanMooney]

i changed my approch in the patches slightly i added gnocchi as a required project in the zuul jobs that use it. This will have zuul prepare a copy of the git repo and copy it to the ci vm automatically like all the other repos and add it to LIBS_FROM_GIT so devstack will know to do a git install https://review.opendev.org/c/openstack/telemetry-tempest-plugin/+/872350 and then in the devstack plugin made it depend on that patch https://review.opendev.org/c/openstack/ceilometer/+/872332

the advantage of this approach is you should be able to do a depend on against a gnocci PR form a ceilometer change. ceilometer was previously installing from git master anyway using pip for quite a while.

if there is a new release then we could drop it form required_project to swap back to the pypi install.

the ohter advantage fo this approch si since we are having zull cache and prepare the git repo for us we are less likely to have a job failure becuase the pip git clone fails due to a connectivity issue with github.

so if you want to wait for a 5.0 then that should be fine.

i was just fixing this because one of my coworkers wanted to look at a devstack env to compare how gnocci and ceilometer were deploy so i have adressed there usecase in my exisiting patches already.

[tobias-urdin]

Released 4.4.3 and 4.5 (branched out stable/4.5 from master) that would include more recent code changes.