search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
70.47k stars 29.41k forks source link

google-cloud-pubsub requires grpcio >= 1.38.1 but HA is pinned to 1.31.0 #53427

Closed g1bs0nsg closed 3 years ago

g1bs0nsg commented 3 years ago

The problem

I have Travis CI setup and attempted a test of Home Assistant 2021.7.4 and it failed with the following:

`INFO:homeassistant.util.package:Attempting install of scapy==2.4.5 INFO:homeassistant.util.package:Attempting install of aiodiscover==1.4.0 Fatal error while loading config: (grpcio 1.31.0 (/home/travis/virtualenv/python3.8.7/lib/python3.8/site-packages), Requirement.parse('grpcio<2.0dev,>=1.38.1'), {'google-cloud-pubsub'}) Failed config General Errors:

Doing some digging I found this: PR to Pin gRPC to 1.31.0

And then found this require grpcio >= 1.38.1 to prevent a possible crash that was merged 23 days ago.

What is version of Home Assistant Core has the issue?

All versions

What was the last working version of Home Assistant Core?

Any version installed prior to the google-cloud-pubsub change

What type of installation are you running?

Home Assistant Core

Integration causing the issue

Google Nest

Link to integration documentation on our website

https://www.home-assistant.io/integrations/nest/

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

probot-home-assistant[bot] commented 3 years ago

nest documentation nest source (message by IssueLinks)

probot-home-assistant[bot] commented 3 years ago

Hey there @allenporter, mind taking a look at this issue as its been labeled with an integration (nest) you are listed as a codeowner for? Thanks! (message by CodeOwnersMention)

allenporter commented 3 years ago

I can take a look at upgrading grpcio and cloud pubsub.

allenporter commented 3 years ago

Hi @g1bs0nsg g1bs0nsg My impression is that home assistant is pinned to an older google-cloud-pubsub release taht is not including the change you referenced about a higher grpcio version. Are you using the home assistant python venv or something else?

allenporter commented 3 years ago

Aside: I went to look at upgrading, but I don't have an arm box right now to reproduce the rpcio issue that was originally raised.

g1bs0nsg commented 3 years ago

Hi @allenporter When I look at my running Home Assistant install which is 2021.5.5 in a venv I have google-cloud-pubsub 2.4.1 installed, and grpcio 1.31.0 This installation went fine when I upgraded a couple of months ago. I tried modifying my travis.yml with pip3 install homeassistant==2021.5.5 which I had a passing build with 2 months ago and it fails with the same message posted above. It's also still installing the same versions of python-nest 4.1.0 and google-nest-sdm 0.2.12 that are installed in my running build.

There are other users experiencing the same, see this thread

The travis build runs in Ubuntu Bionic 18.04.5 LTS, my installation is on Debian 10.

When checking dependencies on google-cloud-pubsub 2.4.1 it comes back with:

(test) root@hass:/srv/test# johnnydep google-cloud-pubsub==2.4.1 2021-07-24 18:16:26 [info ] init johnnydist [johnnydep.lib] dist=google-cloud-pubsub==2.4.1 parent=None 2021-07-24 18:16:28 [info ] init johnnydist [johnnydep.lib] dist=google-api-core[grpc]<2.0.0dev,>=1.22.2 parent=google-cloud-pubsub==2.4.1 2021-07-24 18:16:30 [info ] init johnnydist [johnnydep.lib] dist=grpc-google-iam-v1<0.13dev,>=0.12.3 parent=google-cloud-pubsub==2.4.1 2021-07-24 18:16:32 [info ] init johnnydist [johnnydep.lib] dist=libcst>=0.3.10 parent=google-cloud-pubsub==2.4.1 2021-07-24 18:16:33 [info ] init johnnydist [johnnydep.lib] dist=proto-plus>=1.7.1 parent=google-cloud-pubsub==2.4.1 2021-07-24 18:16:35 [info ] init johnnydist [johnnydep.lib] dist=google-auth<2.0dev,>=1.25.0 parent=google-api-core[grpc]<2.0.0dev,>=1.22.2 2021-07-24 18:16:37 [info ] init johnnydist [johnnydep.lib] dist=googleapis-common-protos<2.0dev,>=1.6.0 parent=google-api-core[grpc]<2.0.0dev,>=1.22.2 2021-07-24 18:16:38 [info ] init johnnydist [johnnydep.lib] dist=grpcio<2.0dev,>=1.29.0 parent=google-api-core[grpc]<2.0.0dev,>=1.22.2

So maybe the issue is that google-cloud-pubsub isn't pinned and it's trying to install the latest version 2.6.1 which has the grpcio requirement?

allenporter commented 3 years ago

Odd, google-cloud-pubsub is pinned but to an older version:

From https://github.com/home-assistant/core/blob/dev/requirements_all.txt#L703 its pinned to

google-cloud-pubsub==2.1.0

I made that change 9 months ago in https://github.com/home-assistant/core/commit/6b57ad9f28a0cf01c60c4f1722c76590e6cb9142

allenporter commented 3 years ago

Also python-google-nest-sdm has a looser but compatible requirement of google-cloud-pubsub>=2.1.0 in its setup.py

So where is google-cloud-pubsub 2.4.1 coming from given that doesn't match home assistants pins?

g1bs0nsg commented 3 years ago

Not sure, there is nothing else installed other than Home Assistant. I don't have any custom components or anything like that. I have integrations for Nest thermostat, and I have several Google Home devices, and I use Google calendar. That's it.

Here is everything installed in my venv:

` Package Version

absl-py 0.12.0 acme 1.13.0 aiodiscover 1.4.0 aiodns 2.0.0 aioesphomeapi 2.6.6 aioharmony 0.2.7 aiohttp 3.7.4.post0 aiohttp-cors 0.7.0 aiohue 2.3.1 aiounifi 26 APScheduler 3.6.3 astral 2.2 astunparse 1.6.3 async-dns 1.1.10 async-timeout 3.0.1 async-upnp-client 0.16.2 asyncio-mqtt 0.8.1 atomicwrites 1.4.0 attrs 20.3.0 awesomeversion 21.2.3 bcrypt 3.1.7 boto3 1.17.57 botocore 1.20.57 broadlink 0.17.0 cachetools 4.2.1 casttube 0.2.1 certifi 2020.12.5 cffi 1.14.5 chardet 4.0.0 ciso8601 2.1.3 click 7.1.2 click-datetime 0.2 colorlog 4.8.0 convertdate 2.3.2 cryptography 3.3.2 cycler 0.10.0 Cython 0.29.23 dataclasses 0.6 decorator 5.0.7 defusedxml 0.6.0 dill 0.3.3 distro 1.5.0 dm-tree 0.1.6 dnspython 2.1.0 DoorBirdPy 2.1.0 ecdsa 0.14.1 emoji 1.2.0 envs 1.3 future 0.18.2 gast 0.3.3 gin-config 0.4.0 google-api-core 1.26.3 google-api-python-client 1.6.4 google-auth 1.29.0 google-auth-httplib2 0.1.0 google-auth-oauthlib 0.4.4 google-cloud-bigquery 2.13.1 google-cloud-core 1.6.0 google-cloud-pubsub 2.4.1 google-crc32c 1.1.2 google-nest-sdm 0.2.12 google-pasta 0.2.0 google-resumable-media 1.2.0 googleapis-common-protos 1.53.0 greenlet 1.1.0 grpc-google-iam-v1 0.12.3 grpcio 1.31.0 gTTS 2.2.2 h11 0.12.0 h5py 2.10.0 ha-ffmpeg 3.0.2 hass-nabucasa 0.43.0 hijri-converter 2.1.1 holidays 0.11.1 home-assistant-frontend 20210504.0 homeassistant 2021.5.5 homeassistant-pyozw 0.1.10 httpcore 0.13.3 httplib2 0.19.1 httpx 0.18.0 idna 2.10 ifaddr 0.1.7 importlib-resources 5.1.2 Jinja2 2.11.3 jmespath 0.10.0 josepy 1.8.0 kaggle 1.5.12 Keras-Preprocessing 1.1.2 kiwisolver 1.3.1 korean-lunar-calendar 0.2.1 libcst 0.3.18 lxml 4.6.3 Markdown 3.3.4 MarkupSafe 1.1.1 matplotlib 3.4.1 multidict 5.1.0 mutagen 1.45.1 mypy-extensions 0.4.3 mysqlclient 2.0.3 netdisco 2.8.2 numpy 1.20.2 oauth2client 4.0.0 oauthlib 3.1.0 opencv-python-headless 4.5.1.48 opt-einsum 3.3.0 packaging 20.9 paho-mqtt 1.5.1 pandas 1.2.4 Pillow 8.1.2 pip 20.2.4 PlexAPI 4.5.1 plexauth 0.0.6 plexwebsocket 0.0.13 promise 2.3 proto-plus 1.18.1 protobuf 3.15.8 psutil 5.8.0 py-cpuinfo 8.0.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycares 3.1.1 PyChromecast 9.1.2 pycocotools 2.0.1 pycognito 2021.3.1 pycparser 2.20 PyDispatcher 2.0.5 pyenvisalink 4.0 pyfttt 0.3 pyHS100 0.3.5.2 PyJWT 1.7.1 PyMeeus 0.5.11 PyNaCl 1.3.0 pynut2 2.1.2 pyOpenSSL 20.0.1 pyotp 2.3.0 pyparsing 2.4.7 PyQRCode 1.2.1 pyRFC3339 1.1 pyroute2 0.5.18 PySocks 1.7.1 python-dateutil 2.8.1 python-didl-lite 1.2.6 python-jose 3.2.0 python-nest 4.1.0 python-openzwave-mqtt 1.4.0 python-slugify 4.0.1 python-telegram-bot 13.1 pytz 2021.1 pywemo 0.6.3 PyYAML 5.4.1 requests 2.25.1 requests-oauthlib 1.3.0 requests-toolbelt 0.9.1 rfc3986 1.4.0 rokuecp 0.8.1 rsa 4.7.2 ruamel.yaml 0.15.100 s3transfer 0.4.2 scapy 2.4.5 scipy 1.4.1 sentencepiece 0.1.95 setuptools 49.2.1 six 1.15.0 slixmpp 1.7.0 sniffio 1.2.0 snitun 0.20 SQLAlchemy 1.4.13 sseclient-py 1.7 tensorboard 2.5.0 tensorboard-data-server 0.6.0 tensorboard-plugin-wit 1.8.0 tensorflow 2.3.0 tensorflow-addons 0.12.1 tensorflow-datasets 4.2.0 tensorflow-estimator 2.3.0 tensorflow-hub 0.12.0 tensorflow-metadata 0.30.0 tensorflow-model-optimization 0.5.0 termcolor 1.1.0 text-unidecode 1.3 tf-models-official 2.3.0 tf-slim 1.1.0 tornado 6.1 tqdm 4.60.0 typeguard 2.12.0 typing-extensions 3.7.4.3 typing-inspect 0.6.0 tzlocal 2.1 uritemplate 3.0.1 urllib3 1.26.4 voluptuous 0.12.1 voluptuous-serialize 2.4.0 Werkzeug 1.0.1 wheel 0.36.2 wrapt 1.12.1 xmltodict 0.12.0 yarl 1.6.3 zeroconf 0.29.0 zm-py 0.5.2 `

allenporter commented 3 years ago

I am able to reproduce this with the following:

$ mkdir ha_test
$ cd ha_test
$ python3 --version
Python 3.8.10
$ python3 -m venv .
$ source bin/activate
(venv) $ python3 -m pip install wheel
(venv) $ pip3 install homeassistant
$ pip3 freeze | grep google-cloud-pubsub
$ hass

At this point I navigated to port 8123 and installed nest integration and get the error:

    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (grpcio 1.31.0 (/home/allen/ha_test/lib/python3.8/site-packages), Requirement.parse('grpcio<2.0dev,>=1.38.1'), {'google-cloud-pubsub'})

Then inspecting dependencies in the venv I see the same thing as reported above:

$ pip3 freeze | grep google-cloud-pubsub
google-cloud-pubsub==2.6.1
allenporter commented 3 years ago

I am not familiar with how home assistants package loading on the fly works, but I assume what happens is that home assistant installed google-nest-sdm which has these requirements:

"requires_dist":[
  "aiohttp (>=3.7.3)",
  "google-auth (>=1.22.0)",
  "google-auth-oauthlib (>=0.4.1)",
  "google-cloud-pubsub (>=2.1.0)",
  "requests-oauthlib (>=1.3.0)"]
]

Then its grabbing the latest version, but not using the versions pinned by requirements_all.txt. Maybe Adding an explicit requirement for nest the nest integration on the same pinned version as the google cloud pubsub integration will do it, but not sure if that is allowed: https://github.com/home-assistant/core/blob/922eec09098e0462b64bea91210a674512b8530a/homeassistant/components/google_pubsub/manifest.json#L5

https://github.com/home-assistant/core/blob/68bf6194e122ef012422e8bb031c602c32763f5f/homeassistant/components/nest/manifest.json#L7

allenporter commented 3 years ago

I've sent a PR which may help, but likely the best thing to do is move forward on grpcio when we can verify the arm problems are fixed.

Nenodema commented 3 years ago

I've sent a PR which may help, but likely the best thing to do is move forward on grpcio when we can verify the arm problems are fixed.

Thanks Allen, do you have any updates on this issue?

I noticed this problem while testing the 2021.8.0 update.

image

allenporter commented 3 years ago

Hi. I'm not sure why you're picking up an unversioned google-cloud-pubsub. I sent a fix to update the nest component to depend on the version pinned by home assistant otherwise... That is, I assumed this was fixed already.

Can you run the pip commands I requested above to help diagnose?

g1bs0nsg commented 3 years ago

I'm still seeing this as well.... I ran through your steps above:

gibsonsg@hass-test:~$ source bin/activate
(gibsonsg) gibsonsg@hass-test:~$ python3 --version
Python 3.8.11
(gibsonsg) gibsonsg@hass-test:~$ pip3 freeze | grep google-cloud-pubsub
(gibsonsg) gibsonsg@hass-test:~$ hass

Navigate to host:8123 and install Nest integration and see the same error you saw above:

raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.ContextualVersionConflict: (grpcio 1.31.0 (/home/gibsonsg/lib/python3.8/site-packages), Requirement.parse('grpcio<2.0dev,>=1.38.1'), {'google-cloud-pubsub'})

Check the venv and get this:

(gibsonsg) gibsonsg@hass-test:~$ pip3 freeze | grep google-cloud-pubsub
google-cloud-pubsub==2.7.0

Don't know if it is helpful or not, but here is the full output from the time I started home assistant until I installed Nest integration:

(gibsonsg) gibsonsg@hass-test:~$ hass Unable to find configuration. Creating default one in /home/gibsonsg/.homeassistant 2021-08-07 09:56:17 WARNING (MainThread) [homeassistant.bootstrap] Waiting on integrations to complete setup: zeroconf 2021-08-07 09:59:16 ERROR (MainThread) [haffmpeg.core] FFmpeg fails [Errno 2] No such file or directory: 'ffmpeg' Traceback (most recent call last): File "/home/gibsonsg/lib/python3.8/site-packages/haffmpeg/core.py", line 136, in open self._proc = await self._loop.run_in_executor(None, proc_func) File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run result = self.fn(*self.args, *self.kwargs) File "/usr/local/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/local/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'ffmpeg' 2021-08-07 09:59:16 WARNING (MainThread) [haffmpeg.tools] Error starting FFmpeg. 2021-08-07 09:59:41 ERROR (MainThread) [aiohttp.server] Error handling request Traceback (most recent call last): File "/home/gibsonsg/lib/python3.8/site-packages/aiohttp/web_protocol.py", line 422, in _handle_request resp = await self._request_handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/aiohttp/web_app.py", line 499, in _handle resp = await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/aiohttp/web_middlewares.py", line 119, in impl return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/security_filter.py", line 60, in security_filter_middleware return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/forwarded.py", line 77, in forwarded_middleware return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/request_context.py", line 24, in request_context_middleware return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/ban.py", line 78, in ban_middleware return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/auth.py", line 144, in auth_middleware return await handler(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/view.py", line 135, in handle result = await result File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/config/config_entries.py", line 128, in post return await super().post(request) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/http/data_validator.py", line 63, in wrapper result = await method(view, request, args, **kwargs) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/helpers/data_entry_flow.py", line 72, in post result = await self._flow_mgr.async_init( File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/data_entry_flow.py", line 153, in async_init flow, result = await task File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/data_entry_flow.py", line 171, in _async_init flow = await self.async_create_flow(handler, context=context, data=data) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/config_entries.py", line 696, in async_create_flow integration.get_platform("config_flow") File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/loader.py", line 498, in get_platform cache[full_name] = self._import_platform(platform_name) File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/loader.py", line 503, in _import_platform return importlib.import_module(f"{self.pkg_path}.{platform_name}") File "/usr/local/lib/python3.8/importlib/init.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 961, in _find_and_load_unlocked File "", line 219, in _call_with_frames_removed File "", line 1014, in _gcd_import File "", line 991, in _find_and_load File "", line 975, in _find_and_load_unlocked File "", line 671, in _load_unlocked File "", line 843, in exec_module File "", line 219, in _call_with_frames_removed File "/home/gibsonsg/lib/python3.8/site-packages/homeassistant/components/nest/init.py", line 11, in from google_nest_sdm.google_nest_subscriber import GoogleNestSubscriber File "/home/gibsonsg/lib/python3.8/site-packages/google_nest_sdm/google_nest_subscriber.py", line 15, in from google.cloud import pubsub_v1 File "/home/gibsonsg/lib/python3.8/site-packages/google/cloud/pubsub_v1/init.py", line 17, in from google.cloud.pubsub_v1 import types File "/home/gibsonsg/lib/python3.8/site-packages/google/cloud/pubsub_v1/types.py", line 37, in from google.pubsub_v1.types import pubsub as pubsub_gapic_types File "/home/gibsonsg/lib/python3.8/site-packages/google/pubsub_v1/init.py", line 17, in from .services.publisher import PublisherClient File "/home/gibsonsg/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/init.py", line 16, in from .client import PublisherClient File "/home/gibsonsg/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/client.py", line 42, in from .transports.base import PublisherTransport, DEFAULT_CLIENT_INFO File "/home/gibsonsg/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/transports/init.py", line 19, in from .base import PublisherTransport File "/home/gibsonsg/lib/python3.8/site-packages/google/pubsub_v1/services/publisher/transports/base.py", line 36, in client_library_version=pkg_resources.get_distribution( File "/home/gibsonsg/lib/python3.8/site-packages/pkg_resources/init.py", line 466, in get_distribution dist = get_provider(dist) File "/home/gibsonsg/lib/python3.8/site-packages/pkg_resources/init.py", line 342, in get_provider return working_set.find(moduleOrReq) or require(str(moduleOrReq))[0] File "/home/gibsonsg/lib/python3.8/site-packages/pkg_resources/init.py", line 886, in require needed = self.resolve(parse_requirements(requirements)) File "/home/gibsonsg/lib/python3.8/site-packages/pkg_resources/init.py", line 777, in resolve raise VersionConflict(dist, req).with_context(dependent_req) pkg_resources.ContextualVersionConflict: (grpcio 1.31.0 (/home/gibsonsg/lib/python3.8/site-packages), Requirement.parse('grpcio<2.0dev,>=1.38.1'), {'google-cloud-pubsub'})

As a test, I cleared out my venv and started over... installed home assistant and then before starting for the first time did:

(ha_test) gibsonsg@hass-test:~/ha_test$ pip3 install google-cloud-pubsub==2.1.0

then started home assistant and installed Nest integration and got no errors.

Likewise, I wiped out the venv again and started fresh. After installing home assistant, if I check lib/python3.8/site-packages/homeassistant/package_constraints.txt I see the entry there for grpcio, so I added one for google-cloud-pubsub:

# gRPC 1.32+ currently causes issues on ARMv7, see:
# https://github.com/home-assistant/core/issues/40148
grpcio==1.31.0

# google-cloud-pubsub, any version newer than 2.1.0 requires grpcio>=1.38.0
google-cloud-pubsub==2.1.0

then started home assistant for the first time and installed Nest integration and also got no errors as it installed 2.1.0 as expected:

(ha_test) gibsonsg@hass-test:~/ha_test$ pip3 freeze | grep google-cloud-pubsub
google-cloud-pubsub==2.1.0
allenporter commented 3 years ago

Thanks for supplying the workaround steps. I'm not sure where the unpinned version of cloud-pubsub is coming from. Maybe the "right" path forward is to just resolve the grpcio pin by fixing the issue on am, so i'll take a look at that.

g1bs0nsg commented 3 years ago

Thanks for supplying the workaround steps. I'm not sure where the unpinned version of cloud-pubsub is coming from. Maybe the "right" path forward is to just resolve the grpcio pin by fixing the issue on am, so i'll take a look at that.

I would agree with that, thank you for your work and assistance on this.

Just a note for anyone else experiencing this... I upgraded my actual running instance to 2021.8.3, and before starting again downgraded google-cloud-pubsub to 2.1.0 from the installed 2.4.1 version and everything seems to be running perfectly fine.

Steps were:

systemctl stop homeassistant
sudo -u homeassistant -H -s
source /srv/homeassistant/bin/activate
pip3 install --upgrade homeassistant
pip3 install --upgrade google-cloud-pubsub==2.1.0
systemctl start homeassistant
allenporter commented 3 years ago

Given https://github.com/grpc/grpc/issues/26094 i worry the grpcio issue is still there, which may mean the latest grpcio doesn't work either.

allenporter commented 3 years ago

I see https://github.com/home-assistant/core/pull/50605 and https://github.com/home-assistant/core/pull/51289 which are relevant.

I'll go ahead and update package_constraints.txt. My previous PR just didn't hit the right file.

g1bs0nsg commented 2 years ago

Hi @allenporter it looks like this issue is back again. A new version of google-api-core has been released which requires grpcio >=1.33.2 and HA is attempting to install that one when it installs google-cloud-pubsub. The new version it is attempting to install is 2.0.0, but version 1.31.2 is the newest one that will work, it requires grpcio >=1.29.0. If I explicitly install 1.31.2 prior to starting HA for the first time then my build is successful.