The local API has changed

[tonybfox]

It looks like google have changed the api as I'm getting 403 Forbidden for anything other than /setup/eureka_info and that now return a public_key

[rithvikvibhu]

Thanks for letting me know! Will take a look at this in a few days. Hopefully, it's just an auth system that's easy to replicate.

[stboch]

I was digging in a bit to see if I can figure out what is going on... it seems that google has moved local communication to port 8443. I tried to do a little MitM but it seems that google is doing some application cert validation. (Either that or something else I did is wrong) They also changed it to require a client ssl for authentication.

[klimov-gett]

@stboch i think you're on point regarding application certificate:

i figure :8443 was google's port for SSL so i did a GET to https://192.168.1.219:8443/setup/bluetooth/status (since auto-connecting to bt speakers is what i do with this api) as a result i received "Error: Peer certificate cannot be authenticated with given CA certificates"

after disabling certifcate validation i got 401, which reads "similar to 403, but in this case authorization is possible"

[cmorche]

What's odd for me is that this is still all working on 8008, and I'm even part of the preview program... This is the first time learning of the local API, so I guess I shouldn't bother spending too much time mucking about.

[ludeeus]

It looks like the change has rolled back, every endpoint I have now tested returns the expected result 🎉

[raddacle]

It looks like the change has rolled back, every endpoint I have now tested returns the expected result 🎉

You're right! Though, I had to reboot my GH devices for it to revert

[jamieshaw]

I've been seeing this flip flop between available and unavailable for the past few days. Not entirely sure what's triggering it…

[Leatherface75]

I think they are preparing some sorts of authorization for higher security and maybe because of closing down NEST API and move it to the Cloud instead.

[rithvikvibhu]

So I finally got my hands on a GH and some updates.

First, the simple way of logging network traffic (with PacketCapture) didn't work because it's a different port and other problems. Tried setting up proxies with burp and mitmproxy, but it wasn't possible to forward other ports with ProxyDroid (even with root). There is also SSL pinning, but Frida was able to bypass that. Finally, I set up a hotspot and started to capture with Wireshark, but during all this, my GH reverted back and I didn't have a chance to capture anything :/ Must have been because of one of the reboots. So yay that it's working right now, but auth may be back at any time. We'll just have to wait till then, I guess.

Simultaneously, I was going through the latest decompiled app and found some interesting bits. Most of the requests had an extra header cast-local-authorization-token. This is likely what we need to get. Unfortunately, the code is obfuscated and I couldn't find what this field should contain or how it is obtained. Will keep trying.

There are also more details on /setup/eureka_info than the docs, including a local-authorization-token-hash or something similar. Again, I didn't get to try this because it reverted back to not requiring any token.

Also, not sure since when, but there's a lot of Nest libraries now. Including Weave stuff (maybe https://openweave.io, haven't really checked it out yet).

That's it for now. If someone knows about the token or has any more info, feel free to comment here. I'm leaving this issue open.

[jkpe]

@rithvikvibhu All of my Google Home Mini in the UK are in a state where they return a 403, if I can help in debugging this let me know

curl -I http://bedroom-ghome.lan:8008/setup/eureka_info
HTTP/1.1 403 Forbidden
Content-Length:0

[rithvikvibhu]

Hey @6a61636b. That's interesting. /setup/eureka_info is the only one everyone could access when other endpoints returned 403. Can you try /setup/bluetooth/status?

[jkpe]

curl -I http://bedroom-ghome.lan:8008/setup/bluetooth/status
HTTP/1.1 403 Forbidden
Content-Length:0

According to the Google Home app it is running System firmware version: 156414, Cast firmware version: 1.40.156414

[thorleifjacobsen]

I can confirm the issue here in Norway on both a Chromecast Ultra and Chromecast 3 and the old Chromecast (not key fob). They all this morning returned 403. After a reboot they started working again.While they were at 403 I could not see any http data going through wireshark when using google home app. Everything went through 8443 and 8009.

I have the pcap file from my chromecast while it had 403 error on google home app. Android -> chromecast ultra configuring and rebooting once if that helps. Would like to send it privately if you need it.

Current Cast Version where API Works "build_version":"156414","cast_build_revision":"1.40.156414"

[Sennevds]

curl -I http://bedroom-ghome.lan:8008/setup/bluetooth/status
HTTP/1.1 403 Forbidden
Content-Length:0

According to the Google Home app it is running System firmware version: 156414, Cast firmware version: 1.40.156414

You need to use the small i. So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

[thorleifjacobsen]

You need to use the small i. So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

Captial i is to get the "headers only" lowercase is to include the HTTP response. He just want to see the headers (403 forbidden) so capital i is perfect.

[Sennevds]

You need to use the small i. So curl -i http://bedroom-ghome.lan:8008/setup/bluetooth/status

Captial i is to get the "headers only" lowercase is to include the HTTP response. He just want to see the headers (403 forbidden) so capital i is perfect.

That's true but still if I use: `PC51:~$ curl -I http://192.168.0.5:8008/setup/bluetooth/status

HTTP/1.1 405 Method Not Allowed
Access-Control-Allow-Headers:Content-Type
Cache-Control:no-cache
Content-Length:0`

and if use: `curl -i http://192.168.0.5:8008/setup/bluetooth/status

HTTP/1.1 200 OK
Access-Control-Allow-Headers:Content-Type
Cache-Control:no-cache
Content-Length:114
Content-Type:application/json
{"audio_mode":1,"connected_devices":[],"connecting_devices":[],"discovery_enabled":false,"scanning_enabled":false}`

So definitely a difference

[thorleifjacobsen]

True, you need to add -X GET when using -I to specify method. Whereas -i somehow adds that manually. Wierd.

Well spotted though

[alertedsnake]

Nothing weird about this. The help explains:

-i, --include       Include protocol response headers in the output
-I, --head          Show document info only

A GET request is the default with curl, and using -i will modify that request. -I means to make a HTTP HEAD request. So the 405 response is not surprising if the endpoint only responds to GET

[rithvikvibhu]

@thorleifjaocbsen, how did you capture the packets? And where did you see the requests over ports 8009 and 8443? I'm asking this because normally http proxies won't be able to see them.

I'd love to take a look at that pcap file. Please send it to rithvikvibhu@gmail.com, if possible.

[thorleifjacobsen]

I have a AP connected to a switch, then I have that port mirrored to my computer (lan adapter) and did dump all traffic going from and to the AP. My andorid is connected to WiFi the Chromecast is connected via ethernet.

10.0.0.9 is my Android Device 10.0.0.8 is my Chromecast Device

I'm connecting using Google Home -> Configuring my Chromecast -> Changing some settings -> Rebooting it -> done pcap.

Filter: ip.addr == 10.0.0.8 or ip.addr == 10.0.0.9 Filter 2: (ip.src == 10.0.0.9 and ip.dst == 10.0.0.8) or (ip.dst == 10.0.0.9 and ip.src == 10.0.0.8)

Not sure if it helps. Sent you the file now :)

[Leatherface75]

What's the command to set an alarm/timer? I can only see commands for lists and delete.

[rithvikvibhu]

@thorleifjaocbsen Thanks for the file! There's not much to find there. 8443 uses TLS so no luck there. Any idea what those TCP packets on port 8009 are?

My GH is again back to sending 403 so I'll try some stuff with the android app.

[rithvikvibhu]

What's the command to set an alarm/timer? I can only see commands for lists and delete.

@Leatherface75 There's no way to set new alarms/timers locally afaik.

[poma]

Mine are back to 403 too except the ones that have Preview Program turned on, looks like those have different firmware update schedule

[rithvikvibhu]

Hey everyone,

I got the API working over port 8443! I'll keep this short.

Turns out the extra header I mentioned in the previous comment was needed. I kept digging in the app, but couldn't find what the value should be. Tried another method with frida and got the token value. Tried this value with postman, and it worked. /setup/bluetooth/status works and others should too, haven't tested. As of now, I have no idea how often or when this token changes or expires, and from where this token is generated. After looking into this, I'll mostly post a guide or script to get it. For those curious, the token is part of Google's new Homegraph and you might be able to find it with root.

[rithvikvibhu]

Okay, created a gist with info: https://gist.github.com/rithvikvibhu/1a0f4937af957ef6a78453e3be482c1f

Please try it out and see if it works.

[rithvikvibhu]

Also, I can't test a few cases. People with multiple rooted phones (with different Google accounts signed in), can you post the first and last few characters of the token? I'm curious if it's different for each home member.

[klimov-gett]

@rithvikvibhu thanks a bunch, tried the Frida method - works like a charm. However, i think it's worth adding that you need (or at least i need) to pass --insecure option to curl for the request to go through. Reason for that being that the certificate installed on my Home Mini has no corresponding root CA / cert chain installed on my device's trust store and hence my Home is treated as a potential thread and curl aborts the request unless that option is passed.

[rithvikvibhu]

Hah yes, thanks @klimov-gett. I'll add that bit.

[Leatherface75]

That "public_key" from eureka_info is that different from that token you are talking about?

[rithvikvibhu]

@Leatherface75 That's the public key for a keypair for GH. The token is different. It is used to authenticate requests made from the app to the GH.

[directman66]

Is it possible to generate a token without a rooted phone yet?

[Leatherface75]

This type om things normally need rooted devices. I think it depends on if Google will offer creating keys for their API or not.

[rithvikvibhu]

@directman66 not yet.

[Drakulix]

For anyone wanting a workaround for their home-assistant setup, if have written a small authentication proxy (you still need your token): https://github.com/Drakulix/ghlapi_proxy

Just point home-assistant to the IP of the proxy instead.

As of now, I have no idea how often or when this token changes or expires, and from where this token is generated.

The token seems to change each time "your home" changes. I had to grab it again after:

• Adding a new device (new sonos speaker, so unrelated to my google home mini)
• Requesting a new sync (although nothing had actually changed) via: "Hey google, sync my devices".

Okay, created a gist with info: https://gist.github.com/rithvikvibhu/1a0f4937af957ef6a78453e3be482c1f

Your decodeProtoFile.js-script did not work for me at first, because it errored out on some devices of mine. I later noticed, those are my sonos speakers linked to my account, which do not have one of the indices of val2 the script is expecting.

Just adding a try-catch block around the second lambda, correctly logged the token of all my devices: https://gist.github.com/Drakulix/e175ab49a416b298345d12eb555d6c24

[rithvikvibhu]

@Drakulix sweet! I don't use home-assistant now, will keep yours in mind.

Just a few questions.

The token seems to change each time "your home" changes.

Huh. Does the old token still work after you get the new one? What is "Sync my devices" really meant to do?

Do all your devices have different tokens? Maybe even non-GH devices (like Sonos) have tokens but at a different location.

I know that the file contains sensitive info to share, but can you try either formatting it with protoc (cat homegraph.pb | protoc --decode_raw) or opening it with a text editor? I just want to know if there's anything that resembles a token in the Sonos object.

[Leatherface75]

Sync devices is like it sounds. Syncing devices so they have all informations including phones, tablets etc. If you add a new lamp or anything else connected to Google Home you need to do that.

[jamieshaw]

@Drakulix Nice one on the proxy – works a treat. One issue though, and I'll open it on the repo is there isn't a simple answer? Is there a way to run this multiple times for multiple devices without some network wizardry spoofing an IP for each docker container as an endpoint for each Google Home device? For the record; I have six… 🤔

(In short, there is no port option in the Google Home component within Home Assistant)

[EDIT: …or the logical answer would be to create a temporarily custom_component when I've got five minutes to make the port a configuration parameter]

[OleksandrBerchenko]

My 5 cents:

According to this instruction I went to file system, copied the file and successfully extracted the token. But the token DID NOT work. Then I noticed that the file was modified on July 13. I opened my Google Home application, waited for a few seconds and refreshed the file: it has been changed. Copied it again and extracted the token: it was different and actually worked.

Two conclusions:

1. I didn't do anything special with my devices since July 13, but the token changed.
2. The phone don't get a new token automatically, unless you manually run Google Home application.

[Drakulix]

The token seems to change each time "your home" changes.

Huh. Does the old token still work after you get the new one? What is "Sync my devices" really meant to do?

No, the old token does not work anymore, that is how I noticed that. "Sync my devices" starts a new request to your providers to register all connected devices with home-graph. This is sometimes necessary especially with home-assistant if my google home is not picking up new devices correctly.

Do all your devices have different tokens? Maybe even non-GH devices (like Sonos) have tokens but at a different location.

All my other devices (two Android TV Boxes) do have tokens as well, although I am not aware of any http-intefaces. The Sonos ones might have some tokens as well, but they do not support the local-api, so I do not really care.

I know that the file contains sensitive info to share, but can you try either formatting it with protoc (cat homegraph.pb | protoc --decode_raw) or opening it with a text editor? I just want to know if there's anything that resembles a token in the Sonos object.

I do not see anything interesting, but here is one of my sonos speakers (with removed id-like structures and names):

Sonos-object protobuf decoded

  7 {
    1 {
      1: "(uuid)"
      2 {
        1: "sonos-(five digits+letters)"
        2: "RINCON_(17 digits+letters)"
      }
    }
    4: "(device name)"
    6: "action.devices.types.SPEAKER"
    7: "action.devices.traits.MediaInitiation"
    7: "action.devices.traits.TransportControl"
    7: "action.devices.traits.Volume"
    7: "action.devices.traits.MediaState"
    7: "action.devices.traits.Assistant"
    7: "action.devices.traits.InputSelector"
    17 {
      1: "Sonos"
    }
    18 {
      2: "(room name)"
      3: "Sonos"
    }
    19: (13 digits)
    20 {
      1 {
        1: "availableInputs"
        2 {
          6 {
            1 {
              5 {
                1 {
                  1: "key"
                  2 {
                    3: "antenna"
                  }
                }
              }
            }
          }
        }
      }
      1 {
        1: "canReportMediaState"
        2 {
          4: 1
        }
      }
      1 {
        1: "commandOnlyOnOff"
        2 {
          4: 0
        }
      }
      1 {
        1: "commandOnlyVolume"
        2 {
          4: 0
        }
      }
      1 {
        1: "levelStepSize"
        2 {
          2: 0x4014000000000000
        }
      }
      1 {
        1: "mediaInitiationAvailableProviders"
        2 {
          6 {
            1 {
              3 {
                8: 0x6f69646152756363
              }
            }
            1 {
              3: "Amazon Music"
            }
            1 {
              3: "AMPed"
            }
            1 {
              3: "Anghami"
            }
            1 {
              3: "Audible"
            }
            1 {
              3: "Bandcamp"
            }
            1 {
              3: "Classical Archives"
            }
            1 {
              3: "Deezer"
            }
            1 {
              3: "Gaana"
            }
            1 {
              3: "Google Play Music"
            }
            1 {
              3: "iHeartRadio"
            }
            1 {
              3: "JUKE"
            }
            1 {
              3: "KKBOX"
            }
            1 {
              3: "MeloMe"
            }
            1 {
              3: "Mixcloud"
            }
            1 {
              3: "My Cloud Home"
            }
            1 {
              3: "NhacCuaTui"
            }
            1 {
              3: "Pandora"
            }
            1 {
              3: "Plex"
            }
            1 {
              3 {
                10: 114
                13: 0x63696e6f6870656d
              }
            }
            1 {
              3: "Qobuz"
            }
            1 {
              3: "QQ\351\237\263\344\271\220"
            }
            1 {
              3: "Radio Javan"
            }
            1 {
              3: "RadioPlay"
            }
            1 {
              3: "rova"
            }
            1 {
              3: "Saavn"
            }
            1 {
              3: "SoundCloud"
            }
            1 {
              3: "Stingray Music"
            }
            1 {
              3: "TIDAL"
            }
            1 {
              3: "TuneIn"
            }
            1 {
              3: "WEYV"
            }
            1 {
              3: "YouTube Music"
            }
            1 {
              3: "\345\207\244\345\207\260FM"
            }
            1 {
              3: "\345\215\203\345\215\203\351\237\263\344\271\220(\345\216\237\347\231\276\345\272\246\351\237\263\344\271\220)"
            }
            1 {
              3: "\345\226\234\351\251\254\346\213\211\351\233\205FM"
            }
            1 {
              3: "\347\275\221\346\230\223\344\272\221\351\237\263\344\271\220"
            }
            1 {
              3: "Spotify"
            }
          }
        }
      }
      1 {
        1: "mediaInitiationByDescription"
        2 {
          4: 1
        }
      }
      1 {
        1: "mediaInitiationCanPlayAudio"
        2 {
          4: 1
        }
      }
      1 {
        1: "needsAssistantUnlink"
        2 {
          4: 1
        }
      }
      1 {
        1: "onDemandAssistantDiscoverability"
        2 {
          4: 1
        }
      }
      1 {
        1: "orderedInputs"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportActionState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportActivityState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportApplicationState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportIsPlayingMediaState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportPlaybackState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportTimeSyncState"
        2 {
          4: 1
        }
      }
      1 {
        1: "supportedMediaTypes"
        2 {
          6 {
            1 {
              3: "Language"
            }
            1 {
              3: "MusicRecording"
            }
            1 {
              3: "MusicAlbum"
            }
            1 {
              3: "MusicGroup"
            }
            1 {
              3: "MusicPlaylist"
            }
            1 {
              3: "Person"
            }
            1 {
              3: "RadioChannel"
            }
            1 {
              3: "RadioBroadcastService"
            }
            1 {
              3: "Thing"
            }
          }
        }
      }
      1 {
        1: "transportControlSupportedCommands"
        2 {
          6 {
            1 {
              3: "NEXT"
            }
            1 {
              3: "PREVIOUS"
            }
            1 {
              3: "PAUSE"
            }
            1 {
              3: "STOP"
            }
            1 {
              3: "RESUME"
            }
            1 {
              3: "SEEK_RELATIVE"
            }
          }
        }
      }
      1 {
        1: "volumeCanMuteAndUnmute"
        2 {
          4: 1
        }
      }
      1 {
        1: "volumeDefault"
        2 {
          2: 0x4024000000000000
        }
      }
      1 {
        1: "volumeMaxLevel"
        2 {
          2: 0x4059000000000000
        }
      }
    }
    26 {
      1: "(google account mail)"
    }
    34 {
      1: 1
    }
    16: "Sonos"
  }

@Drakulix Nice one on the proxy – works a treat. One issue though, and I'll open it on the repo is there isn't a simple answer? Is there a way to run this multiple times for multiple devices without some network wizardry spoofing an IP for each docker container as an endpoint for each Google Home device? For the record; I have six… thinking

Yes there is, if your docker-host is connected via ethernet to your home network, you may setup a macvlan (or ipvlan) network via docker and assign that to all your proxy containers. Every container should then get its own mac or ip address.

(In short, there is no port option in the Google Home component within Home Assistant)

Yeah I noticed that too, total bummer.

2. The phone don't get a new token automatically, unless you manually run Google Home application.

Can confirm that, I always have to open the google home app on my rooted device before I can extract new tokens. This makes automatic extraction rather annoying.

[Leatherface75]

Maybe something that can upload that file once a day or when it see that file changed or something to server there it extracts and updates the token? A better solution would be something that can communicate with Google Home API and get tokens from there. https://developers.google.com/actions/smarthome/reference/rest/

Everyone should also complain to Google for doing this like users did to Logitech for example in a similar situation. https://www.home-assistant.io/blog/2018/12/17/logitech-harmony-removes-local-api/

[Leatherface75]

I am getting this error after it finds first google home token.

/tmp/decodeProtoFile.js:24 var token = getObjByKey(val2['7'], '28')['28']; ^

TypeError: Cannot read property '28' of undefined at /tmp/decodeProtoFile.js:24:57 at Array.forEach () at /tmp/decodeProtoFile.js:21:18 at Array.forEach () at Object. (/tmp/decodeProtoFile.js:19:6) at Module._compile (internal/modules/cjs/loader.js:774:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:785:10) at Module.load (internal/modules/cjs/loader.js:641:32) at Function.Module._load (internal/modules/cjs/loader.js:556:12) at Function.Module.runMain (internal/modules/cjs/loader.js:837:10)

[Leatherface75]

If i change this line var token = getObjByKey(val2['7'], '28')['28']; to var token = getObjByKey(val2['7'], '28');

then i am able to get tokens for all Google Home devices but it shows lot of other things from Google Assistant too like lamps, consoles, tvboxes, androidtv etc.

Device: undefined, Taklampa Vardagsrum Token: undefined

Device: Google Home Mini, Högtalare Kök Token: { '28': 'TOKEN-REMOVED'}

Device: Google Home Mini, Högtalare Sovrum Token: { '28': 'TOKEN'-REMOVED'}

Device: undefined, MiBox3 Sovrum Token: undefined

Device: undefined, RPi3 Sovrum Token: undefined

I think it just needs a google assistant/home integration that gets the tokens from there. Home Assistant for example already have that and are using their API so i think it should be able to get tokens with updated code.

[ASchneiderBR]

A better solution would be something that can communicate with Google Home API and get tokens from there. https://developers.google.com/actions/smarthome/reference/rest/

Hello sir! I was thinking about that as well. Wouldn't be easier to connect to the device using those API over the internet? Not sure if there is an API available for that or how to identify the correct Google Home Mini device you need to get the alarm information from.

I am a Home Assistant user and already have configured a test app needed to expose my devices to the GH API and control them.

Sorry, if can not help you guys solve this problems since I lack the proper coding skills...

Thanks!

[Leatherface75]

That home_graph file contains that information and the API is in that link. I also use Home Assistant and all my things from Home Assistant is in that home_graph file. I think that's why that script fails because it finds a lot of other things. If Home Assistant integration gets that home_graph file it can get all tokens from that file and use it for connecting like earlier.

[Leatherface75]

As a workaround for you guys with more than 1 google home and Linux or similiar i modified the proxy code above and added a number and compiled in 3 versions. Open main.rs file in src dir and you will see those variables in the top.

Then i did this

ip addr add 192.168.0.120/24 dev enp2s0 ip addr add 192.168.0.121/24 dev enp2s0 ip addr add 192.168.0.122/24 dev enp2s0 export SOURCE_IP_1=192.168.0.15 export SOURCE_IP_2=192.168.0.30 export SOURCE_IP_3=192.168.0.70 export LISTEN_IP_1=192.168.0.120 export LISTEN_IP_2=192.168.0.121 export LISTEN_IP_3=192.168.0.122 export TOKEN_1=TOKEN_FOR_GOOGLE_HOME_1 export TOKEN_2=TOKEN_FOR_GOOGLE_HOME_2 export TOKEN_3=TOKEN_FOR_GOOGLE_HOME_3 ghlapi_proxy_1 & ghlapi_proxy_2 & ghlapi_proxy_3 &

Then change your configuration.yaml file if you're using Home Assistant with the new IP's Change IP's and name of network card for your setup.

[Leatherface75]

and it didn't took long time and tokens was changed so this method doesn't work if you don't want to update your token once or a few times a day.

[KapJI]

@rithvikvibhu did you try to figure out how app is obtaining these tokens? As their lifespan is quite short we need another way than getting token from a rooted device.

[Leatherface75]

Home assistant for example is communicating and have their own token for Google assistant. Shouldnt it be possible to get information with updated addon there?

[Leatherface75]

Installed Android on a virtual machine with Home app open and then a crontab that grabs that file from that android virtual machine once in an hour. Let's see how that works been working for almost one day now.