search

home-assistant / core

:house_with_garden: Open source home automation that puts local control and privacy first.
https://www.home-assistant.io
Apache License 2.0
72.63k stars 30.4k forks source link

Google Cloud TTS integration failing to load in 0.115 #40148

Closed andynbaker closed 4 years ago

andynbaker commented 4 years ago

The problem

I've used the Google Could TTS integration for all my TTS needs for a few months now without issue. I recently switched to 0.115 beta and did not notice any issues with this integration (that I recall). However, the integration now fails to load at startup, each time reporting the same error in the logs. The google_cloud_say service also fails, even though I can reach my google homes through a change volume service call, for example.

Environment

I'm running Home Assistant Container in Docker on Raspberry Pi OS

arch | armv7 dev | false docker | true hassio | false installation_type | Home Assistant Container os_name | Linux os_version | 5.4.51-v7l+ python_version | 3.8.5 timezone | America/Chicago version | 0.115.0b11 virtualenv | false

Problem-relevant configuration.yaml

tts:
  - platform: google_cloud
    key_file: googlecloud.json

Traceback/Error logs

Logger: homeassistant.config
Source: components/google_cloud/tts.py:7
First occurred: 11:01:33 AM (1 occurrences)
Last logged: 11:01:33 AM

Platform error: tts
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config.py", line 815, in async_process_component_config
    platform = p_integration.get_platform(domain)
  File "/usr/src/homeassistant/homeassistant/loader.py", line 401, in get_platform
    cache[full_name] = importlib.import_module(
  File "/usr/local/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 783, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/usr/src/homeassistant/homeassistant/components/google_cloud/tts.py", line 7, in <module>
    from google.cloud import texttospeech
  File "/usr/local/lib/python3.8/site-packages/google/cloud/texttospeech.py", line 19, in <module>
    from google.cloud.texttospeech_v1 import TextToSpeechClient
  File "/usr/local/lib/python3.8/site-packages/google/cloud/texttospeech_v1/__init__.py", line 21, in <module>
    from google.cloud.texttospeech_v1.gapic import text_to_speech_client
  File "/usr/local/lib/python3.8/site-packages/google/cloud/texttospeech_v1/gapic/text_to_speech_client.py", line 22, in <module>
    import google.api_core.gapic_v1.client_info
  File "/usr/local/lib/python3.8/site-packages/google/api_core/gapic_v1/__init__.py", line 18, in <module>
    from google.api_core.gapic_v1 import config
  File "/usr/local/lib/python3.8/site-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
    import grpc
  File "/usr/local/lib/python3.8/site-packages/grpc/__init__.py", line 23, in <module>
    from grpc._cython import cygrpc as _cygrpc
ImportError: Error relocating /usr/local/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so: backtrace: symbol not found

Additional information

I'm in over my head here, but googling led me to a few other examples of this issue: https://github.com/zeromq/pyzmq/issues/1323 https://github.com/grpc/grpc/issues/6126

andynbaker commented 4 years ago

I just pulled down 0.115b1 and do not see the issue anymore. Just tested with a service call to be sure and can in fact cast a google cloud tts call to my google home minis. So i know for sure the issue was not in 0.115b1.

fancy90 commented 4 years ago

me too 114 work nomal , 115 beta i check config google cloud error

Platform error tts.google_cloud - Error relocating /usr/local/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so: backtrace: symbol not found

pavax commented 4 years ago

I just updated to the birthday release: 0.115.0 and struggling now with the very same exception. The mentioned exception shows up on startup in the log-file and every time I hit the "check configuration" button.

key value
arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7l
python_version 3.8.5
supervisor 245
timezone Europe/Zurich
version 0.115.0
virtualenv false
wtreur commented 4 years ago

Same issue on my end. Running in a docker container in Pi4

arch | armv7l dev | false docker | true hassio | false installation_type | Home Assistant Container os_name | Linux os_version | 4.19.97-v7l+ python_version | 3.8.5 timezone | Europe/Amsterdam version | 0.115.0 virtualenv | false

alastaid commented 4 years ago

Same issue here, 0.114 was ok, showed up in 0.115

arch armv7l
chassis  
dev false
docker true
docker_version 19.03.12
hassio true
host_os Raspbian GNU/Linux 10 (buster)
installation_type Home Assistant Supervised
os_name Linux
os_version 5.4.51-v7l+
python_version 3.8.5
supervisor 245
timezone Europe/London
version 0.115.1
virtualenv false
artbrayko commented 4 years ago

Same

alastaid commented 4 years ago

115.2 doesnt fix it for me either, I would have thought there would be a lot of people using google TTS, hence this would be getting a lot more focus, which makes me wonder if I just need to change my yaml entry somehow, going back to the docs....

rosenbaek commented 4 years ago

Same issue for me.

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7l
python_version 3.8.5
supervisor 245
timezone Europe/Copenhagen
version 0.115.2
virtualenv false
dr3men commented 4 years ago

And me...

Mincka commented 4 years ago

I can also confirm I have the same error when checking the configuration. I had to disable the integration (commented out).

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7l
python_version 3.8.5
supervisor 245
timezone Europe/Paris
version 0.115.2
virtualenv false
andynbaker commented 4 years ago

I fired up a copy of HA on HA OS and do not see the issue. Only when running in container do I see it.

Edit: this is on the 5.2 dev build of HA OS

makingwilliam commented 4 years ago

That's not true, I am using Home Assistant on HassOS under the current Hass.io build and have been experiencing the same problem.

The problem

Google Cloud TTS is not loading and/or cannot be enabled after upgrading to 0.115.1 and after updating to 0.115.2 and 0.115.3.

Environment

component environment
arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7
python_version 3.8.5
supervisor 245
timezone America/New_York
version 0.115.2
virtualenv false

Problem-relevant configuration.yaml

tts:
  - platform: google_cloud
    key_file: google_cloud.json
    voice: en-GB-Wavenet-B
    time_memory: 43200

Traceback/Error logs

When checking configuration after enabling integration through YAML config, the following error is displayed:

Platform error tts.google_cloud - Error relocating /usr/local/lib/python3.8/site-packages/grpc/_cython/cygrpc.cpython-38-arm-linux-gnueabihf.so: backtrace: symbol not found

Only found the following in my logs:

2020-09-20 08:29:45 INFO (SyncWorker_37) [homeassistant.loader] Loaded google_cloud from homeassistant.components.google_cloud
2020-09-20 08:29:45 INFO (SyncWorker_37) [homeassistant.loader] Loaded google_cloud from homeassistant.components.google_cloud

Additional information

andynbaker commented 4 years ago

Ahh my apologies, I'm running the dev branch of HA OS (5.2). So I correct my statement, I do not see the error on HAOS 5.2

Mincka commented 4 years ago

Updated to HassOS 5.2 (from HassOS 4.13) and the problem is still there. Seems unrelated.

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 5.2
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7l
python_version 3.8.5
supervisor 245
timezone Europe/Paris
version 0.115.2
virtualenv false
andynbaker commented 4 years ago

To be sure it is indeed now working for me, I just called the service from developer tools. The message came through fine using my usual cloud voice profile. The only change on my end was going from Container to Home Assistant OS 5.2. If it helps, I'm booting from an SSD, so my eeprom had been updated. But I was booting from an SSD on container so I'm not sure that's the special sauce.

So as of now, I'm on 5.2, running 0.115.2, have no more TTS issues in my logs, and can successfully make a TTS service call where I couldn't before.

Mincka commented 4 years ago

Could it be related to https://github.com/home-assistant/core/pull/39893 ?

Mincka commented 4 years ago

Related issue (current version): https://github.com/home-assistant/core/issues/40266 Already happened in the past (no answers): https://github.com/home-assistant/core/issues/27343

makingwilliam commented 4 years ago

Is there a code owner for this integration? I noticed that the issue hasn't been assigned to anyone yet, but it is affecting multiple people.

Noahma3 commented 4 years ago

I am having the same issue.

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.12
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7l
python_version 3.8.3
supervisor 245
timezone America/Denver
version 0.114.3
virtualenv false
harsanyi76 commented 4 years ago

This bug seems to be Raspberry OS/Arm related. Google TTS still works on x86/Ubuntu/Docker

caiocaminoski commented 4 years ago

Same problem here!

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7
python_version 3.8.5
supervisor 245
timezone Europe/Zurich
version 0.115.2
virtualenv false
andynbaker commented 4 years ago

Updated to HassOS 5.2 (from HassOS 4.13) and the problem is still there. Seems unrelated.

arch armv7l chassis embedded dev false docker true docker_version 19.03.11 hassio true host_os HassOS 5.2 installation_type Home Assistant OS os_name Linux os_version 4.19.127-v7l python_version 3.8.5 supervisor 245 timezone Europe/Paris version 0.115.2 virtualenv false

I see everyone with this issue on HaasOS is running the 32bit version of either 4.19 or 5.2. I'm no longer seeing the issue after upgrading to the 64bit version of HaasOS 5.2. This seems like the most glaring difference between my install and others?

arch aarch64
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 5.2
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v8
python_version 3.8.5
supervisor 245
timezone America/Chicago
version 0.115.2
virtualenv false
alastaid commented 4 years ago

@Mincka looking at https://github.com/home-assistant/core/issues/40266 it does look relevant, especially as it looks like the issue is 32bit. I am new to github projects, what is protocol here do we add in the developer on that PR to this thread and see if we can get him to look at it and get it moving, I have spent a ton of time on my Google TTS automations and am missing them.

Mincka commented 4 years ago

I think you meant #39893. I saw that the author was in holiday on Twitter so we may not @ him and leave him enjoy his vacation while we are patient. :smile: Personally, I just fell back to the basic TTS (Google Translate). The voice is not as good as Google Cloud TTS but it does the job. I think it's a reasonable workaround.

alastaid commented 4 years ago

Thanks for the guidance, I must confess I didn't realise I can fall back to Google Translate, that will be fine for me and I can wait till its fixed.

mariwing commented 4 years ago

The same issue is also affecting other Google components like the PubSub. I don't think there is an workaround for that so I would say that this needs to be fix as soon as possible.

andynbaker commented 4 years ago

Does anyone else with a Pi4 have an appetite for trying out the 64bit dev version to verify the issues are gone in that build?

wtreur commented 4 years ago

Sure. I don’t mind giving it a go. Which container/tag should I use?

andynbaker commented 4 years ago

Sure. I don’t mind giving it a go. Which container/tag should I use?

I saw the bug beginning somewhere around 0.115.b4 or so, running Container in Docker on 32 bit Raspberry Pi OS on a Pi4. I switched to the 64bit dev build 5.2 of Home Assistant OS on the same Pi4 and no longer see the issue (I currently get TTS messages cast over my google home minis, no problem).

So in my mind there are at least two things worth trying: 1) do the same as me and load up a 64 bit 5.2 dev build of Home Assistant OS, or 2) maybe try loading up some other 64 bit OS of your choosing and see if the stable build of Home Assistant Container 0.115 no longer shows the issue.

geochilmaru commented 4 years ago

In my case, I've changed encoding setting from linear16 to mp3 or ogg_opus and then it works now. my version is 0.115.2.

Mincka commented 4 years ago

In my case, I've changed encoding setting from linear16 to mp3 or ogg_opus and then it works now. my version is 0.115.2.

The encoding was not specified in my case so mp3 already. I tried the other settings, it did not work (= not able to check the configuration / reload the module). Did you have the same error message than the OP?

geochilmaru commented 4 years ago

In my case, I've changed encoding setting from linear16 to mp3 or ogg_opus and then it works now. my version is 0.115.2.

The encoding was not specified in my case so mp3 already. I tried the other settings, it did not work (= not able to check the configuration / reload the module). Did you have the same error message than the OP?

Sorry. It might not help you. I had a problem with google cloud tts but not the same issue. My problem is as follows. It means what I did is just a temporary solution, then it is still not normal.

[homeassistant.components.websocket_api.http.connection.139898363333840] 'ko-KR-Wavenet-B' not a Frame instance Traceback (most recent call last): File "/usr/src/homeassistant/homeassistant/components/websocket_api/commands.py", line 137, in handle_call_service await hass.services.async_call( File "/usr/src/homeassistant/homeassistant/core.py", line 1315, in async_call task.result() File "/usr/src/homeassistant/homeassistant/core.py", line 1350, in _execute_service await handler.func(service_call) File "/usr/src/homeassistant/homeassistant/components/tts/init.py", line 167, in async_say_handle url = await tts.async_get_url( File "/usr/src/homeassistant/homeassistant/components/tts/init.py", line 340, in async_get_url filename = await self.async_get_tts_audio( File "/usr/src/homeassistant/homeassistant/components/tts/init.py", line 367, in async_get_tts_audio data = self.write_tags(filename, data, provider, message, language, options) File "/usr/src/homeassistant/homeassistant/components/tts/init.py", line 470, in write_tags tts_file["artist"] = artist File "/usr/local/lib/python3.8/site-packages/mutagen/_file.py", line 74, in setitem self.tags[key] = value File "/usr/local/lib/python3.8/site-packages/mutagen/id3/_tags.py", line 339, in setitem raise TypeError("%r not a Frame instance" % tag) TypeError: 'ko-KR-Wavenet-B' not a Frame instance

jacekpaszkowski commented 4 years ago

Same issue after upgrading to 1.115.2 from 1.114

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7
python_version 3.8.5
supervisor 245
timezone Europe/Warsaw
version 0.115.2
virtualenv false
Mincka commented 4 years ago

@geochilmaru: Ok, that's what I thought. Maybe you should create a specific issue with your workaround.

fancy90 commented 4 years ago

0.115.3 not fix it yet so sad

wtreur commented 4 years ago

Sure. I don’t mind giving it a go. Which container/tag should I use?

I saw the bug beginning somewhere around 0.115.b4 or so, running Container in Docker on 32 bit Raspberry Pi OS on a Pi4. I switched to the 64bit dev build 5.2 of Home Assistant OS on the same Pi4 and no longer see the issue (I currently get TTS messages cast over my google home minis, no problem).

So in my mind there are at least two things worth trying: 1) do the same as me and load up a 64 bit 5.2 dev build of Home Assistant OS, or 2) maybe try loading up some other 64 bit OS of your choosing and see if the stable build of Home Assistant Container 0.115 no longer shows the issue.

Oh, silly me. 🤦 I can't just simply run a another docker container on my existing OS to check a potentially 64bit related fix when I'm running a 32bit os. I am afraid testing that is going to be hard for me then, since I only have one Pi4 and I am using it for more than just Home Assistant and I don't have a spare sd-card lying around.

Mincka commented 4 years ago

frenck balloob 0.115.3 installed now, still having this issue, has nobody picked up on this really? It's affecting a ton of users as you can see here.

Don't @ people for nothing. Fallback to basic TTS (Google Translate), be patient, reinstall on a 64-bit platform or help to find a solution.

makingwilliam commented 4 years ago

I have to agree, this was all working before a MAJOR release was made and since that release, many, many users have been affected and reporting this issue. The issue has not been assigned to anyone, meaning there probably isn't a code owner. So it appears as if this integration has been abandoned. The whole purpose of running Hass.io on HassOS is to prevent these types of things from happening to the vast majority of users who are not advanced enough to complete the necessary troubleshooting or have the programming skills to make changes on their own to resolve the problem. That said, it would be nice for someone on the Home Assistant team to at least acknowledge this problem and give us some feedback on it or when a fix might be made or if this integration should be removed because it has been abandoned like the rest of the google/nest integrations which have been abandoned for over a year now.

It's sad to know that all these new important features that are always coming out undermine things that were already working with no problem and then when they become broken by these new features, no one can really spare the time to look at them. The go to answer is to just use another TTS or completely reinstall your system in a manner that you didn't initially intend. No, the answer should be do proper testing and when something new breaks something old, focus on how to repair what was already working and shouldn't have been impacted by the new feature(s) to begin with. And also, where is the undo button on an upgrade, or rollback button. That is what we desperately need, after upgrading when you don't like the way something is working, you can click one button and go back to your previous build with ease. Then you can wait for the problem to be resolved before upgrading again.

RestOp commented 4 years ago

Same problem(((

nickrout commented 4 years ago

Guys why don't you look: the code owner is in manifest.json.

makingwilliam commented 4 years ago

@nickrout the question still remains, why hasn't the bot assigned this to the code owner automatically as it normally would. That said, The code owner is listed as @shred86 though it appears @lufton has also authored the code. Not sure if either of you could take a look at this issue or not.

Mincka commented 4 years ago

And also, where is the undo button on an upgrade, or rollback button. That is what we desperately need, after upgrading when you don't like the way something is working, you can click one button and go back to your previous build with ease. Then you can wait for the problem to be resolved before upgrading again.

Downgrade / "rollback" is not so easy...

Use the snapshot feature and this add-on https://github.com/sabeechen/hassio-google-drive-backup if you want to go back to a previous version.

Also, where was you for the "proper testing" during the beta releases?

gannas42 commented 4 years ago

I just encountered this issue as well after upgrading from 0.114.4 to 0.115.3 .

arch armv7l
chassis embedded
dev false
docker true
docker_version 19.03.11
hassio true
host_os HassOS 4.13
installation_type Home Assistant OS
os_name Linux
os_version 4.19.127-v7
python_version 3.8.5
supervisor 245
timezone America/Chicago
version 0.115.3
virtualenv false
lufton commented 4 years ago

@nickrout the question still remains, why hasn't the bot assigned this to the code owner automatically as it normally would. That said, The code owner is listed as @shred86 though it appears @lufton has also authored the code. Not sure if either of you could take a look at this issue or not.

Everything I can tell is it looks like there is something wrong with python-texttospeech library witch is required by this integration and depends on grpc. But I don't have enough time to investigate, I'm sorry.

RestOp commented 4 years ago

Downgrade / "rollback" is not so easy...

Easy sudo ha core update --version=0.114.4

nickrout commented 4 years ago

The code owner is listed as @shred86

No it isn't.

@lufton is there anything any of us can do to help? It looks like this is some sort of library problem in alpine builds on rpi 32? It wouldn't be the first time recently.

aidbish commented 4 years ago

@thewilliambond > No, the answer should be do proper testing and when something new breaks something old, focus on how to repair what was already working and shouldn't have been impacted by the new feature(s) to begin with

So we'll see you in the next round of beta testing helping out with the testing?

aidbish commented 4 years ago

@nickrout i wonder if exporting the tts files and setting up as a custom integration will allow upgrading of dependencies to work out which requires changing. looking at the requirements for python-texttospeech, its 0.4.0 the latest release is up to 2.2 and lots of breaking changes when migrating to 2.0, maybe trying the last before 2.0 maybe worthwhile

nickrout commented 4 years ago

@nickrout i wonder if exporting the tts files and setting up as a custom integration will allow upgrading of dependencies to work out which requires changing. looking at the requirements for python-texttospeech, its 0.4.0 the latest release is up to 2.2 and lots of breaking changes when migrating to 2.0, maybe trying the last before 2.0 maybe worthwhile

I think the problem is that there is no wheel for grcp (which is a dependency of python-texttospeech) so hassio has to build it, and that is failing on rpi 32.

I am guessing here, but grcp is a dependency of a dependency, so I am not sure if it is roped in to the home assistant wheel builds.

aidbish commented 4 years ago

@nickrout grpc seems to be listed in the dependencies in wheels

i check the armv7 one and its listed. https://wheels.home-assistant.io/alpine-3.12/armv7/