again a device linking problem

[NikAWing]

Hello all!

I'm really new to node red and google actions, but I get things done step by step :) After IFTTT became a no-go solution, I'm glad I've found out about node-red-contrib-google-smarthome.

To be honest, setting it up with google is really difficult, I'm glad there is a how-to, though the how-to also has puzzling parts (e.g. "Client ID and Secret: The same strings you generated and entered on Google Search Console earlier", google search console?)

Anyway, I have it running, I can reach it via reqbin, I can reach it when I try to link it in the google home app. If I change the settings from "use google login" to other credentials, it also changes in the "add device" process in google home app. BUT: no matter what I select (and for now I use google login), the device will not be added to google home app. All it says is "Could not reach [test] devicename. Please try again." (I replaced my test device with "devicename")

I looked through the issues (and solved issues) and restartet node red to see the debug messages. But I see no errors.

The moment I click the "Link" button I see this:

HttpAuth:httpAuthRegister(GET /oauth)
GET /oauth?response_type=code&client_id=[...]

The moment I log in via google, I see this:

HttpAuth:httpAuthRegister(GET /oauth)
GET /oauth?response_type=code&client_id=
HttpAuth:httpAuthRegister(POST /oauth): body = {"username":"","password":"","client_id":[etc ...]
HttpAuth:httpAuthRegister(POST /oauth): Google login
HttpAuth:httpAuthRegister(POST /oauth): email [...] valid: true
HttpAuth:handleUserAuth(): login successful
HttpAuth:handleUserAuth(): authCode successful; authCode = [...]
POST /oauth 302 84.697 ms - 1776
HttpAuth:httpAuthRegister(GET /oauth)
GET /oauth?response_type=code&client_id=[...] 200 5.646 ms - 7981

I have attached a debug node to the management node and there nothing appears.

Where should I look at to fix this?

Thanks for hints and with best regards, Nik

[Caprico85]

google search console?

Oops. I meant Google Actions Console. On my main job I have to work with Search Console pretty often. That's why I mixed it up.

Where should I look at to fix this?

I heard there can be problems during linking when no devices exist. Do you have any devices added in Node-RED? If not, try adding a device. A simple switch should do. It doesn't need inputs or outputs, it just needs to be there.

If that doesn't help, try Google Cloud Logs at https://console.cloud.google.com/logs/query.

[NikAWing]

Oh, sorry, forgot to mention that I put the node "Light" on the flow (and connected it to management).

But the google log shows nothing at all, I think that's suspicious :) Oh, could a questionmark in the client ID and an ampersand and 2 exclamation marks in the secret be a problem?

[Caprico85]

Oh, sorry, forgot to mention that I put the node "Light" on the flow (and connected it to management).

Nodes don't need a connection to the management node. Inputs on the nodes are for sending the current state of your device in, outputs are for getting the control commands out.

To be "connected" to their management, nodes need to have the same "SmartHome" in their config.

 

Oh, could a questionmark in the client ID and an ampersand and 2 exclamation marks in the secret be a problem?

This could very well be the problem. We had problems with special characters earlier. I wasn't able to completely fix it yet. So for now, please try with alphanumeric characters only.

[NikAWing]

(Like I said, I just started with all of this ^^ )

Alright, I removed everything non-alphanumeric from ID and secret, updated account lining in actions console, updated both in the management console. Tried to link it with google home app and got an error showing the OLD ID. At the same time errors popped up in the management node debug.

I disabled the test in actions and re-enabled it, the error in home app disappeared but the "could not reach" problem came back. And like before, no debug messages appeared in node red.

Also not a single entry in google log-explorer.

So I think that the connection itself is okay and something else must interfere.

[Caprico85]

The usual device linking process looks like this.

User selects our service in the app, login page is loaded:

2021-09-10T05:36:30.826639032Z HttpAuth:httpAuthRegister(GET /oauth)
2021-09-10T05:36:30.862119145Z GET /oauth?response_type=code&client_id=XXXXX&redirect_uri=XXXXX&scope=openid+offline_access+profile+email&state=XXX 200 29.304 ms - 7981
2021-09-10T05:36:32.153786355Z GET /favicon.ico 404 7.156 ms - 150

User completes login, we give out an auth code to Google:

2021-09-10T05:36:51.153016618Z HttpAuth:httpAuthRegister(POST /oauth): body = {"username":"","password":"","client_id":"XXXXX","redirect_uri":"XXXXX","state":"XXXXX","response_type":"","id_token":"XXXXX"}
2021-09-10T05:36:51.154152860Z HttpAuth:httpAuthRegister(POST /oauth): Google login
2021-09-10T05:36:51.961381883Z HttpAuth:httpAuthRegister(POST /oauth): email someone@someewhere.de valid: true
2021-09-10T05:36:51.963063642Z HttpAuth:handleUserAuth(): login successful
2021-09-10T05:36:51.970128227Z HttpAuth:handleUserAuth(): authCode successful; authCode = 2XxaWTVBMfze87Uz2HkuG3KYYgsoVxPtnfd5Yv112ZDg
2021-09-10T05:36:52.001914461Z POST /oauth 302 929.289 ms - 1804

Google calls our service to exchange the auth code into an access token.

2021-09-10T05:36:53.845580884Z HttpAuth:httpAuthRegister(/token): query = {}
2021-09-10T05:36:53.845986819Z HttpAuth:httpAuthRegister(/token): body = {"grant_type":"authorization_code","code":"XXXXX","redirect_uri":"XXXXX","client_secret":"XXXXX"}
2021-09-10T05:36:53.847938315Z Auth:exchangeAuthCode(): user = XXXXX
2021-09-10T05:36:53.854978265Z HttpAuth:handleAuthCode(): respond success; token = {"token_type":"bearer","access_token":"XXXXX","refresh_token":"XXXXX","expires_in":216000}
2021-09-10T05:36:53.860723537Z POST /token 200 24.716 ms - 168

Google calls our service to request a SYNC to get a list of our devices,, we answer with that list.

2021-09-10T05:36:55.177830772Z HttpActions:httpActionsRegister(/smarthome): request.headers = {"host":"XXXXX:3001","user-agent":"Mozilla/5.0 (compatible; Google-Cloud-Functions/2.1; +http://www.google.com/bot.html)","content-length":"80","accept-encoding":"gzip, deflate, br","authorization":"Bearer XXXXX","content-type":"application/json;charset=UTF-8","google-assistant-api-version":"v1","x-forwarded-for":"XXXXX","x-forwarded-host":"XXXXX:3001","x-forwarded-port":"3001","x-forwarded-proto":"https","x-forwarded-server":"traefik","x-real-ip":"XXXXX"}
2021-09-10T05:36:55.178068583Z HttpActions:httpActionsRegister(/smarthome): request.headers.authorization = Bearer XXXXX
2021-09-10T05:36:55.178140927Z HttpActions:httpActionsRegister(/smarthome): reqdata = {"inputs":[{"intent":"action.devices.SYNC"}],"requestId":"11317413063764181378"}
2021-09-10T05:36:55.179997424Z HttpActions:httpActionsRegister(/smarthome): user: XXXXX
2021-09-10T05:36:55.180204610Z HttpActions:httpActionsRegister(/smarthome): SYNC
2021-09-10T05:36:55.181182155Z HttpActions:_sync()
2021-09-10T05:36:55.183159433Z HttpActions:_sync(): response = [... a long list with all my devices ...]
2021-09-10T05:36:55.187627266Z POST /smarthome 200 11.716 ms - 1989

Google sends a QUERY to query the state of our devices:

2021-09-10T05:36:59.436842434Z HttpActions:httpActionsRegister(/smarthome): request.headers = {"host":"XXXXX:3001","user-agent":"Mozilla/5.0 (compatible; Google-Cloud-Functions/2.1; +http://www.google.com/bot.html)","content-length":"228","accept-encoding":"gzip, deflate, br","authorization":"Bearer XXXXX","content-type":"application/json;charset=UTF-8","google-assistant-api-version":"v1","x-forwarded-for":"XXXXX","x-forwarded-host":"XXXXX","x-forwarded-port":"3001","x-forwarded-proto":"https","x-forwarded-server":"traefik","x-real-ip":"XXXXX"}
2021-09-10T05:36:59.437219567Z HttpActions:httpActionsRegister(/smarthome): request.headers.authorization = Bearer XXXXX
2021-09-10T05:36:59.437352951Z HttpActions:httpActionsRegister(/smarthome): reqdata = {"inputs":[{"intent":"action.devices.QUERY","payload":{"devices":[{"id":"aec7a9bb.661578"},{"id":"cb782baf.ebdc98"},{"id":"e271d04234a4125a"},{"id":"4cb98146.ce54"},{"id":"79ab566d.42c538"}]}}],"requestId":"6700015578589366527"}
2021-09-10T05:36:59.438375861Z HttpActions:httpActionsRegister(/smarthome): user: XXXXX
2021-09-10T05:36:59.438769243Z HttpActions:httpActionsRegister(/smarthome): QUERY
2021-09-10T05:36:59.439799028Z HttpActions:_query()
2021-09-10T05:36:59.441220945Z Device:getStates(): deviceIds = ["aec7a9bb.661578","cb782baf.ebdc98","e271d04234a4125a","4cb98146.ce54","79ab566d.42c538"]
[... some more lines about our devices current state...]
2021-09-10T05:36:59.444506182Z Device:getStates(with-deviceIds): states[deviceId] = {"online":true,"thermostatMode":"heat","thermostatTemperatureSetpoint":20,"thermostatTemperatureAmbient":23.875}
2021-09-10T05:36:59.445664455Z HttpActions:_query(): deviceStates = [... answer with the state of my devices (current temperatures, are lights on or off ...]
2021-09-10T05:36:59.450370828Z POST /smarthome 200 13.636 ms - 515

The first two steps seem to work fine in your case. Do you have output from the other steps too?

Are the URLs in your Google Actions Console set correctly? Your log output only shows output from the Auth URL http://example.com:3001/oauth. In Actions Console there also is a Token URL (http://example.com:3001/token) and a fulfillment URL (http://example.com:3001/smarthome) set. Maybe there is a typo in one of these URLs so Google can't reach it.

[NikAWing]

The urls are looking like this (though I use a different port and https)

If I access the URLs with my browser: /smarthome results in "Cannot GET /smarthome" /oauth results in "response_type undefined must equal "code"" /token results in "invalid client id or secret"

in the management google smarthome settings, I have: jwt key in /home/pi/.node-red/certs/xxxx.json webserver settings path: empty use http Node-RED root path: unchecked ext SSL offload: unchecked public key: /home/pi/.node-red/certs/node-cert.pem private key: /home/pi/.node-red/certs/privatekey.pem

The first 2 steps look okay here, the difference starts with this line of your example:

2021-09-10T05:36:53.845580884Z HttpAuth:httpAuthRegister(/token): query = {}

I see this:

HttpAuth:httpAuthRegister(GET /oauth) GET /oauth?response_type=code&client_id=[...]&redirect_uri=https://oauth-redirect.googleusercontent.com/r/noderedcontrol-44a7b&state=[a lot of chars] 200 10.820 ms - 7981

Then nothing else appears in the log.

Thanks for helping me! :)

[Caprico85]

OK, I admit I have no idea what's happening here.

The problem seems to happen during or after the redirect from the login form to Google.

You can try if the redirect itself happens. Open https://example.com:3001/oauth?client_id=XXXXX&redirect_uri=https%3A%2F%2Foauth-redirect-sandbox.googleusercontent.com%2Fr%2Fproject-id&response_type=code&state=abc in your browser. Replace example.com with your domain, XXXXX with your client id and project-id with your project id (which you can get from the address bar in Google Actions Console). Leave the state value as abc. You should see your login page. Submit it. You should get redirected to https://oauth-redirect.googleusercontent.com/r/project-id?code=YYYYY&state=abc. If you get redirected to anywhere else or not at all, this is the problem. The page you are redirected should show the error message "Invalid State Parameter. The Link can not be completed." (as I havent found out how to get a valid state parameter). Other error messages indicate there might be other problems.

 

Other things I would try:

• Switch to username/password authentication. It's easier to set up and you eliminate possible problems from Google Sign in process.
• Stop Node-RED, delete the file google-smarthome-auth-<some_id>.json from Node-RED's user dir (where your setttings.js, flows.json etc. are) and restart Node-RED. This will remove old tokens, user logins etc.
• Maybe even delete the project in Google Actions console and recreate it to have a "clean" start.

[Caprico85]

For the sake of completeness: Version of node.js, Node-RED and node-red-contrib-google-smarthome? Any reverse proxy or otherwise "special" setup?

[NikAWing]

Hello Caprico85,

I've tried it and I'm being redirected to https://oauth-redirect-sandbox.googleusercontent.com/r/project-id?code=YYYYY&state=abc and the error code is Invalid State Parameter. The Link can not be completed.

The difference I see is that in my URL is an additional "-sandbox"

I tried switching to user/pass before with the same results in the google home app (same "could not reach" error)

No special setup here. Since I just started using a Raspberry Pi for home automation, it's a new Raspberry with updated Raspbian connected via LAN and fix IP, port forwarding set up to it for the smarthome node port. Node red v1.3.4 node.js 10.24.0 node-red-contrib-google-smarthome 0.1.6 and I just did upgrade to 0.1.7

Here is the upgrade log, I see these warning messages for every node I install or upgrade. (except the message for Math.random())

2021-09-19T13:34:35.545Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production node-red-contrib-google-smarthome@0.1.7 2021-09-19T13:34:39.099Z [err] npm 2021-09-19T13:34:39.100Z [err]
2021-09-19T13:34:39.100Z [err] WARN npm npm does not support Node.js v10.24.0 2021-09-19T13:34:39.101Z [err] npm 2021-09-19T13:34:39.101Z [err] WARN npm You should probably upgrade to a newer version of node as we 2021-09-19T13:34:39.102Z [err] npm WARN npm 2021-09-19T13:34:39.102Z [err] can't make any promises that npm will work with this version. 2021-09-19T13:34:39.102Z [err] npm 2021-09-19T13:34:39.102Z [err] WARN npm 2021-09-19T13:34:39.103Z [err] Supported releases of Node.js are the latest release of 4, 6, 7, 8, 9. 2021-09-19T13:34:39.103Z [err] npm 2021-09-19T13:34:39.103Z [err] WARN npm 2021-09-19T13:34:39.103Z [err] You can find the latest version at https://nodejs.org/ 2021-09-19T13:34:52.909Z [err] npm 2021-09-19T13:34:52.910Z [err] WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. 2021-09-19T13:34:55.823Z [out] + node-red-contrib-google-smarthome@0.1.7 2021-09-19T13:34:55.823Z [out] updated 1 package in 15.058s 2021-09-19T13:34:55.857Z rc=0

I'll wait for a reply until I delete the project on google and try again :)

[ckhmer1]

Regarding the Invalid State Parameter error message, the state value contains special characters. Could you please share one of the value you get?

[NikAWing]

chkmer1 and Caprico85, please bother with me a little longer, I'm really a little lost because this is completely new to me and IMO setting up everything on the different google sites still is puzzling to me.

I did what Caprico85 suggested above again. I opened https://example.com:3001/oauth?client_id=XXXXX&redirect_uri=https%3A%2F%2Foauth-redirect-sandbox.googleusercontent.com%2Fr%2Fproject-id&response_type=code&state=abc, logged in via google account, and was redirected to the page with error "Invalid State Parameter. The Link can not be completed." There is nothing else on this page, no state value. Do you mean the characters in the URL of this error page? "code=...&state=abc ?

[Caprico85]

To make setup easier I am thinking about doing an introduction video where I go through the whole setup step by step. Anybody knows a good (free) screen recording software where I can put text boxes and blur out parts of the screen? It doesn't need to be able to record from webcam, as you don't want to see or hear me ;-)

Now, back to the problem:

The message "Invalid State Parameter. The Link can not be completed." is exactly what I was expecting. It shows the redirect back to Google is working. Google shows this message because we didn't include a correct state parameter. But at leat Google shows something which means our redirect to Google works.

There is something you could check. Can you please have a look at Google Actions Console. How does the menu in the Develop tab looks like? Like this?

Or like this?

The second one is the expected one. But somehow @jazzgil in #188 managed to get the other menu. And now he isn't able to continue, almost like in your case. I just want to check if you have the same problem.

[ckhmer1]

Are you able to include my latest pull request? It should help to understand the problem.

[Caprico85]

Are you able to include my latest pull request?

Done. Released as 0.1.11.

[ChutneyMary]

This link is a basic solution for creating very short explanatory videos -

https://www.screentogif.com

On Sat, 9 Oct 2021 at 20:34, Caprico @.***> wrote:

To make setup easier I am thinking about doing an introduction video where I go through the whole setup step by step. Anybody knows a good (free) screen recording software where I can put text boxes and blur out parts of the screen? It doesn't need to be able to record from webcam, as you don't want to see or hear me ;-)

Now, back to the problem:

The message "Invalid State Parameter. The Link can not be completed." is exactly what I was expecting. It shows the redirect back to Google is working. Google shows this message because we didn't include a correct state parameter. But at leat Google shows something which means our redirect to Google works.

There is something you could check. Can you please have a look at Google Actions Console. How does the menu in the Develop tab looks like? Like this?

[image: image] https://user-images.githubusercontent.com/2081806/136655723-682ceed5-b5da-40e0-aba5-4523ce8e7ce9.png

Or like this?

[image: image] https://user-images.githubusercontent.com/2081806/136655739-40066d82-4561-4437-8397-9950c27160a9.png

The second one is the expected one. But somehow @jazzgil https://github.com/jazzgil in #188 https://github.com/mikejac/node-red-contrib-google-smarthome/issues/188 managed to get the other menu. And now he isn't able to continue, almost like in your case. I just want to check if you have the same problem.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mikejac/node-red-contrib-google-smarthome/issues/180#issuecomment-939290011, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH3MQTBIIC5ID442SH6RJ43UGAZELANCNFSM5DOWHUIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[ChutneyMary]

I've noticed the last couple of releases (0.1.11 and the previous release) don't show up in the Node-Red palette manager as an 'update available'. Has something changed in the update method?

I'm currently on 0.1.8 and no updates are showing.

On Sat, 9 Oct 2021 at 20:46, Caprico @.***> wrote:

Are you able to include my latest pull request?

Done. Released as 0.1.11.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/mikejac/node-red-contrib-google-smarthome/issues/180#issuecomment-939291603, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH3MQTEKHR3KUXQWGV3KMOLUGA2SXANCNFSM5DOWHUIQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[Paul-Reed]

Shows as a update here...

Maybe a question for the node-RED forum.

[ChutneyMary]

Interesting... If I go to the 'Install' tab, and search for the node, it has the following displayed. If I'm on the 'Nodes' tab, I only get the 'in use' displayed.

[ckhmer1]

You have to go in the "Nodes" tab and search for the google smarthome. You should get the update button

[ChutneyMary]

I'm familiar with the Update button, having used it many times before. It's just not showing up anymore with 0.1.8 installed.

I'm tempted to delete all instances of google-smarthome and try re-installing it from the Install tab. This is going to be a real pain to set up again though. I've also tried re-booting.

Is there any other fault-finding suggested before I need to uninstall and reinstall?

[Paul-Reed]

You're taking this issue off-topic, and would be much better asking about your problem in the node-RED forum, as failing to update a node is unlikely to be a smarthome problem. I wouldn't uninstall & reinstall, just ask in the forum, and we'll give you some suggestions to try.

[Paul-Reed]

@Caprico85 ActivePresenter seems to provide those features. It's commercial software, but all the features are unlocked even if you don't buy a licence. However, it does put a watermark top right (it's not too obtrusive). File output format is selectable, and includes mp4.

[NikAWing]

@Caprico85 It looks like the 2nd picture, I have Invocation, Actions and Account linking there

I'll install the newest release now.

[Caprico85]

I am working on a new setup guide with screenshots. Currently it covers the normal setup, but excludes advanced topics like Google Sign-In or the new local execution. You can find it here

A video would be nice, but is much more work than I thought and it makes it almost impossible for others to contribute (as you can't send a PR for a video.

[ckhmer1]

Is this issue still present?

[NikAWing]

right now I still have no connection, but I didn't try the guide you posted 9 days ago yet, Just updated all node red nodes in my palette, I'll try it in a few hours this afternoon and see if I can set it up this time :)

[NikAWing]

alright, I again got to the point where I log in. and it still says "could not reach ..." everything is fine using the urls and described tests, just google home has problems :| the management node has a yellow dot in front of "ready", no idea if that is important.

If I fill in the data into the \check form textboxes, I get "OK"s everywhere

[NikAWing]

I've updated to node red 2.2.0 and node.js v14, also not relevant to the problem, lol

[NikAWing]

maybe I'm closer to finding the problem now: in debug messages I see this: "GoogleSmartHome: Node-RED is using HTTPS but no local http port was defined, local execution will fail."

Could this be the reason that the app can not connect though the \check website sees no problems?

[Caprico85]

Unfortunately not. The warning message means your Node-RED uses HTTPS but you didn't set a port for local execution.

Local execution (a execution model with less latencies and faster responses from your devices) would not work in your case. But your problem is much earlier, as you are not even able to log in.

I'm currently out of ideas on what's your problem. There is one thing I learned just yesterday. Sometimes there can be problems if you try to log in from your home network (see https://github.com/mikejac/node-red-contrib-google-smarthome/issues/241#issuecomment-1034230208). Can you please disable WiFi on your phone so you are only connected with the mobile network (3G/4G/5G) and try to log in again?

[NikAWing]

I just tried on my phone instead of tablet, but it's the same result / error message :> Could I really be the only one who's not able to connect? D: (in some kind of way that's funny lol)

oh, about the fulfillment port: I connect to node red via https, so I have to enter any port number (and also set a port forward to it?) It won't work if the node red connection is happening through https?

[Paul-Reed]

Any unused port will do. I use port 8880. No need to set a port forward, as it's local traffic.

[NikAWing]

ok, but this didn't change anything either

[FireWizard52]

@NikAWing,

@Caprico85 suggested to disable your Wifi. This way you will sign-in from outside your LAN. This will avoid a NAT Loopback or hairpinning issues with your router. You only said that you changed your device from tablet to phone, but that did not answer the question.

[NikAWing]

ahh, well for me it was kinda logical that phone is not WIFI, but LTE, else I could just use a tablet. So, to make it clear: using internet via telephone provider did not change anything.

To understand it better: the \check website in the how-to, is it google related? I suppose it is just directly accessing the node without any google service being involved, right? IOW: if all tests are ok still could mean that google can not access it because of some problem between google service and the node?

[ckhmer1]

The check page is not using the Google services but it connects directly to the Node-RED. Have you used it from outside your network? Using mobile connection or another internet access?

[NikAWing]

I just tried the check page from smartphone (via SIM card and LTE connection) and (wow, what a hazzle to enter the secret and ID into the fields on smartphone, lol) Every check is OK and the data string appears (payload, dummy, swVersion, hwVersion etc) but no, the google home app says "could not reach [test] device. please try again" after I entered the same user/pass data as I did on the check page

[ckhmer1]

Hi, have you fixed the issue? Have you tried with latest version?

[NikAWing]

hey :) I just updated to 0.4 and set it to MDNS, removed the port in the textbox. google home still gives me the "could not reach" error, the /check test page works, I can enter id/secret, user/pass and see 5 OKs. But google home app, error :/

let me ask again, just to be sure: the debug message "GoogleSmartHome: Node-RED is using HTTPS but no local http port was defined, local execution will fail." is not the problem, right?

[ckhmer1]

I guess that you are not able to link the account with the Google home app, right? If so, forget the local fulfillment, not selecting mDNS neither UDP.

Have you configured an https with real certificate like let's encrypt?

If you can share your external URL, I'll try to check it.

[NikAWing]

finally I have some time to play around :) Ok, so, currently I'm puzzled, I need to check the how-to again.

I just made an SSL cert on the Raspi using openSSL. Then I went to NodeRed and checked the smarthome node again, section "webserver settings", but there I already have entered the 2 PEM files (public key=/home/pi/.node-red/certs/node-cert.pem, private key=/home/pi/.node-red/certs/privatekey.pem") which I've created while trying to set up the smarthome node the 1st time.

I access nodeRed itself via https:// already since I've set it up. Stupid me did not write down every setup step from the beginning, I started to do that a little too late, lol. I don't remember if I made these already present certs using let's encrypt, but I have the same 2 .pem files in my nodeRed settings.js file for https access.

Looking at the how-to, I see that you did not fill out the 2 boxes for public/private key. But the certs are also not mentioned in the how-to, just at the very beginning.

Really, this must must be just one little detail that is missing, lol. Though I seem to be the only one with this problem :/

edit: I've put a device in the flow and selected "sensor", I get some "not found" errors in the debug, but I just started to find out more.

[Caprico85]

I just made an SSL cert on the Raspi using openSSL.

Certificates generated with openSSL won't work as they are self-signed. Google only accepts certificates from a trusted certificate authority Let's Encrypt, ZeroSSL, DigiCert, IdenTrust, ...

I don't remember if I made these already present certs using let's encrypt,

You can check the issuer of your certificates in your browser. Open your (HTTPS encrypted) Node-RED and click on the lock icon in the adress bar. From there you can open the certificate info.

As long as you are able to open Node-RED without any browser warnings or any "I know what I am doing" questions, the certificates should be good.

But the certs are also not mentioned in the how-to, just at the very beginning.

There are too many ways to get certificates. Some people use Let's encrypt, some use other CAs. Some people use certbot to get the certificates, some people use other tools. Some people use a reverse proxy with automatic certificate management. Some people use Node-RED on their NAS with its own admin UI. We can't explain all these ways in our Readme.

Personally I use Caddy as reverse proxy running in front of Node-RED. Caddy fully automates getting and renewing the certificates. With Caddy you don't need to install and configure certbot. You don't have to enter any certificates in Node-RED. You don't need to restart Node-RED after each certificate renewal.

I've written a small guide on how to use Caddy here: https://github.com/mikejac/node-red-contrib-google-smarthome/blob/master/docs/caddy.md

[NikAWing]

so it must be the SSL cert ... I'll check, try and report back :D

[NikAWing]

Caprico85, thanks for being so patient!

I played around with certbot, then installed caddy and: I got it working, it finally connected/linked and I see the "test" sensor I put on the flow. Now it's time to see how everything works :)

MANY THANKS! I learned new things :)

edit: oh, and have a nice 2023! edit2: is there a how-to with google assistant anywhere? I google around since I linked it, I can't see the temperatures in the home app, I can get google home reply with some temperature values, some lead to google results. Anyway, a big step has been achieved since sept 2021 lol

[ChutneyMary]

Regarding setting up Caddy -

I've followed the guide through and haven't had any success. I see an early issue at step 3 of the guide, "Check if the new port is reachable by opening https://192.168.0.100:13001/ in your browser." I've changed the IP address to the correct value but see the following message -

Cannot GET /

What should the Google Smarthome test page look like?

When I revert to my existing method of .pem and .key files (which I'd like to discontinue due to ongoing costs), the working installation shows the 'Cannot Get /' message also (voice commands are working fine).

[Caprico85]

Sorry. Thats a mistake in my instructions. I'll fix that.

The correct URL is http://192.168.0.100:13001/check. The test page either shows "SUCCESS" or the test form.

Getting "Cannot GET /" is still good. If port 13001 was dead, you wouldn't even see this error message.

[ChutneyMary]

Thanks. I've changed the port to 13001 and successfully get to the Google Smarthome test page. I'm not having success with the check after installing Caddy. Can I clarify a couple of things in the How-To:

When editing the Caddyfile, should I assume that the https://example.com:3001 should be changed to my own domain?

ie. https://xyz.ddns.net:3001

Where the line 'reverse_proxy localhost:13001' is, should that be changed to the IP of the host running Google Smarthome?

ie. reverse_proxy 10.1.1.7:13001

I have no idea whether Caddy is actually running. Is there a way to ascertain when/if the Certificate creation has occurred? When I navigate to the Google Smarthome Test Page, I see an error that says "This site can't provide a secure connection. xyz.ddns.net sent and invalid response. ERR_SSL_PROTOCOL_ERROR"

[ChutneyMary]

If I use the following command to start Caddy, I see a response as follows -

Command caddy start --config /etc/caddy/Caddyfile

Response WARN Caddyfile input is not formatted; run the 'caddy fmt' command to fix inconsistencies {"adapter": "caddyfile", "file": "/etc/caddy/Caddyfile", "line": 2} Error: loading initial config: loading new config: starting caddy administration endpoint: listen tcp 127.0.0.1:2019: bind: address already in use Error: caddy process exited with error: exit status 1

Could you please paste in your Caddy config file and BOLD out the areas that need changing? Line 2 in the Config file appears to be the email address.

[Caprico85]

When editing the Caddyfile, should I assume that the https://example.com:3001/ should be changed to my own domain? ie. https://xyz.ddns.net:3001/

Correct.

Where the line 'reverse_proxy localhost:13001' is, should that be changed to the IP of the host running Google Smarthome? ie. reverse_proxy 10.1.1.7:13001

Correct. If Caddy is running on another host as the Smarthome service or in a container, you need to replace localhost with the hostname or IP of the smarthome host.

Is there a way to ascertain when/if the Certificate creation has occurred?

Have a look at Caddy's logfile. How you do this depends on the operating system. docker logs <container name of Caddy> if running with Docker.systemctl status caddy(orsystemctl status -ln1000to see more lines) on Ubuntu. Or maybe there is a logfile at/var/log/caddy.log`.

If I use the following command to start Caddy, I see a response as follows - [...] Error: loading initial config: loading new config: starting caddy administration endpoint: listen tcp 127.0.0.1:2019: bind: address already in use

Either some other service already uses port 2019. Or Caddy is already running as a system service in the background. On Ubuntu, try systemctl status caddy to see if Caddy is already running as a service.

Could you please paste in your Caddy config file and BOLD out the areas that need changing.

I cannot make it bold. Replace everything marked with **

{
    # Replace with your email address
    email **info@example.com**
}

# Replace example.com with your domain.
# Replace 3001 with your external port (the port you set in the Google Console).
https://**example.com**:**3001** {
    # replace localhost with the host where the smarthome service is running
    # Replace 13001 with the port where the smarthome service is running
    reverse_proxy **localhost**:**13001**
}

Or if also running Node-RED behind Caddy:

{
    # Replace with your email address
    email **info@example.com**
}

# Replace with your domain
**example.com** {
    route /* {
        # Replace 13001 with the port where Node-RED is running
        # replace localhost with the host where Node-RED is running
        reverse_proxy **localhost**:**1880**
    }

    # Use password protection if you want
    basicauth * {
        # You can encrypt passwords by running `caddy hash-password`. Alternateively, first bcrypt, then Base64 encode your password.
        myusername
        JDJhJDEwJEh6YW5CNU5zM28zbnF1OHVEWjNySHVGTFRHVVpSY2RyNDJZdUR4TnIvbzhTTWFzZTdmV2Zp
    }
}

# Replace example.com with your domain.
# Replace 3001 with your external port (the port you set in the Google Console).
https://**example.com**:**3001** {
    # replace localhost with the host where the smarthome service is running
    # Replace 13001 with the port where the smarthome service is running
    reverse_proxy **localhost**:**13001**
}

[ChutneyMary]

Just wanted to pass on my thanks to @Caprico85 for your assistance. Caddy is now installed and voice control is working fine.

The use of 'systemctl status caddy' was invaluable in working through a few issues. For what it's worth, the error was initially described as "challenge failed", "could not get certificate from issuer". After a lot of googling, forwarding ports 80 and 443 to the Raspberry Pi hosting Caddy resolved that.

Subsequent errors described in Node-Red -

"Auth:isValidRefreshToken(): refreshToken not found" "HttpAuth:handleRefreshToken(): invalid refresh token"

were resolved by creating a new OAuth 2.0 Client ID in the Google Cloud console.