Domoticz devices not visible in DZGA-flask

Xavier82 commented 7 months ago

I have today installed DZGA-Flask on a Raspberry Pi 4B (8GB), with OS Bullseye, Domoticz 2023.2 (BETA build 15679). This is my test-environment.

I have entered all information correct but the Dashboard doesn't display the devices from Domoticz. I can see under settings\info: Info | admin About DZGA-Flask Version: 23.9 Domoticz Version: 2023.2 (build 15686)

DewGew commented 7 months ago

Have you enabled allow http requests in domoticz?

Xavier82 commented 7 months ago

Yes, it is activated .....

Op 03-12-2023 16:47 schreef DewGew ***@***.***>:

Have you enabled allow http requests in domoticz? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

DewGew commented 7 months ago

Any errors when you sync your devices?

Xavier82 commented 7 months ago

This is in the logging of dzga-flask: `Logs | admin 2023-12-04 08:22:25 Smarthome server has started.

2023-12-04 08:22:25 Running without ssl 2023-12-04 10:32:54 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:32:56 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:32:58 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:32:59 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:32:59 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:48 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:48 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:48 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 240 2023-12-04 10:33:48 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327 2023-12-04 10:33:51 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:51 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327 2023-12-04 10:33:51 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:51 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327 2023-12-04 10:33:55 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:33:55 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327 2023-12-04 10:37:14 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-04 10:37:14 X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327 2023-12-04 10:37:16 Starting new HTTPS connection (1): accounts.google.com:443 2023-12-04 10:37:16 https://accounts.google.com:443 "POST /o/oauth2/token HTTP/1.1" 200 None 2023-12-04 10:37:16 Starting new HTTPS connection (1): homegraph.googleapis.com:443 2023-12-04 10:37:16 https://homegraph.googleapis.com:443 "POST /v1/devices:requestSync HTTP/1.1" 404 None 2023-12-04 10:37:18 Starting new HTTP connection (1): X.X.X.X:80844 2023-12-04 10:37:18 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 327`

In the old DZGA, DZGA collected all devices from Domoticz and showed them in de device tab, even if there were issues with Google connection.

DewGew commented 7 months ago

Try disable google assistant in settings page then click the sync button. Are you using a separate user in domoticz?

Xavier82 commented 7 months ago

no, I use the same user in Domoticz as in dzga-flask, adjusted from default in dzga to the same user in Domoticz...

And yes I also disbaled the google assistant, but no luck

The strange thing is that dzga-flask shows the Domoticz version, so there is connectivity...

Xavier82 commented 7 months ago

As can be seen in screenshot below, strange behaviour: error dzga-flask

So in Settings in DZGA-flask I consciously changed the password of the Domoticz user to a wrong password. Then DZGA-flask tells me "Devices synced with domoticz" while in the logg can be seen there is a Connection Error to Domoticz (because of the wrong password), after the error logg the tells me that de Devices are saved into the admin_devices.json, which also can be seen in the screenshot as empty.

To tackle and investigate the issue I have changed the rights in linux to "0777" (all users can do everything) just to find out if it is a userpermission issue. Despite this still doesn't work.

Xenomes commented 7 months ago

Sorry to interrupt the conversation, but have you checked if the room specified in DZGA has devices in Domoticz?

DewGew commented 7 months ago

Looks like you get 401 Unauthorized response from domoticz. Check your credentials settings

Xavier82 commented 7 months ago

Sorry to interrupt the conversation, but have you checked if the room specified in DZGA has devices in Domoticz?

Thanks for thinking along. I use room "0", this is the same room as I use in the old DZGA environment settings, this had devices.

Xavier82 commented 7 months ago

Looks like you get 401 Unauthorized response from domoticz. Check your credentials settings

Yes thats the thing I wanted to show. I did on purpose changed the password in DZGA-flask with a wrong one. Then I get the 401 error BUT DZGA-flask shows that devices are synced and saved in admin_devices.json file which is empty as can be seen in the screen.

When restoring DZGA-flask with the correct password I don't get the devices sync message and I neither get the 401 error, I get the 200 code....

DewGew commented 7 months ago

Are you using docker?

Xavier82 commented 7 months ago

No, not using donker, Just install on rpi4b 8gb with buster os and Domoticz 2023.2.

Since this is my test environment I was able to reboot all, but no solution 😞

Op 06-12-2023 17:56 schreef DewGew ***@***.***>:

Are you using docker? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

Xenomes commented 7 months ago

Do you have own ssl certificate loaded in Domoticz?

Xavier82 commented 7 months ago

No I don't have a ssl, I use NGROK.

But from my point of view, and experience of old DZGA when connection to Domoticz was active I could see the devices even when no connection to Google.

Op 06-12-2023 20:47 schreef Xenomes ***@***.***>:

Do you have own ssl certificate loaded in Domoticz? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>

DewGew commented 7 months ago

Im using rpi 2b with buster, rpi3b with bullseye and synology DSM6 . All those 3 works for me is there any devices added in "username"_devices.json after syncing device (with Google Assistant disabled in UI)?

Xavier82 commented 7 months ago

None, the file remains empty. I did reinstall but also without luck.

I just don't understand where it goes wrong.

So lets take it step by step: Is my assumption correct? : (since assumption is mother of all fuckups...) In DZGA-flask I go to settings. Under user settings: Domoticz URL: http://IP-ADDRESS_of_Domoticz:PORT_of_Domoticz (Domoticz and DZGA-flask are on the same Rpi4B) Domoticz user: Username of user in Domoticz Domoticz password: Password of the user in Domoticz Domoticz roomplan id: set to "0" (=zero) DZGA password: adjusted password for access to DZGA-flask Google Assistant: Enabled Domoticz Admin: Yes (greyed-out) Auth Token: long string (greyed-out)

Server setting: Actions on Google Client id: 12-digits number I got from Google Actions on Google CLient secret: Oauth2 client secret I got from Google Actions on Google API key: (greyed-out) Temp unit: Celsius Speak Language: Dutch Use SSL: Disabled

File upload: Uploaded the Smart-home-key.json file.

Info shows: DZGA-Flask Version: 23.9 Domoticz Version: 2023.2 (build 15686)

Pressing "Restart Server" and after restart pressing "Sync Devices" shows these entry in logg: 2023-12-07 14:37:38 Smarthome server has started. 2023-12-07 14:37:38 Running without ssl 2023-12-07 14:37:42 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:42 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:42 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:37:42 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:37:43 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:44 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getdevices&plan=0&filter=all&used=true HTTP/1.1" 200 12718 2023-12-07 14:37:47 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:47 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:37:49 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:49 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:37:50 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:50 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:37:50 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:37:50 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:38:27 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:38:27 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:38:30 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:38:30 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:38:31 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:38:31 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:38:31 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:38:31 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:39:02 Settings Saved! 2023-12-07 14:39:02 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:39:02 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:39:02 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:39:02 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:39:05 Starting new HTTPS connection (1): accounts.google.com:443 2023-12-07 14:39:05 https://accounts.google.com:443 "POST /o/oauth2/token HTTP/1.1" 200 None 2023-12-07 14:39:05 Starting new HTTPS connection (1): homegraph.googleapis.com:443 2023-12-07 14:39:05 https://homegraph.googleapis.com:443 "POST /v1/devices:requestSync HTTP/1.1" 404 None 2023-12-07 14:39:11 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:39:11 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:39:11 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:39:11 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:41:21 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:41:21 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:41:27 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:41:27 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:53:41 Starting new HTTPS connection (1): homegraph.googleapis.com:443 2023-12-07 14:53:41 https://homegraph.googleapis.com:443 "POST /v1/devices:requestSync HTTP/1.1" 404 None 2023-12-07 14:53:47 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:47 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:47 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:53:47 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:53:54 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:54 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:53:56 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:56 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:56 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 14:53:56 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 14:53:58 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 14:53:58 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331

I would expect that "Devices" would be populated with devices which are found and retrieved from Domoticz. Since I get 200-status code I assume that there is a connection to Domoticz.

In Domoticz the setting "Allow Basic-Auth authentication over plain HTTP (API only)." is enabled. User sharing is set to "yes" and rights is "admin" in Domoticz.

DewGew commented 7 months ago

if you set Google Assistant: Disabled in UI, we can role out that its a google issue. the click sync devices.

Also you can try run manually in terminal, there might be errors not visable in Ul:

sudo systemctl stop dzga-flask #If service is running
cd /home/${USER}/DZGA-Flask
source env/bin/activate
python3 smarthome.py

Xavier82 commented 7 months ago

So I did:

Google Assistant is Disabled.

This is the logging: `2023-12-07 15:07:19 Smarthome server has started.

2023-12-07 15:07:19 Running without ssl 2023-12-07 15:07:23 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:23 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:23 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 15:07:23 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 15:07:25 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:25 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getdevices&plan=0&filter=all&used=true HTTP/1.1" 200 12697 2023-12-07 15:07:31 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:31 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:31 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 15:07:31 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 2023-12-07 15:07:37 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:37 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 2023-12-07 15:07:37 Starting new HTTP connection (1): X.X.X.X:8084 2023-12-07 15:07:37 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 `

Xavier82 commented 7 months ago

if you set Google Assistant: Disabled in UI, we can role out that its a google issue. the click sync devices.

Also you can try run manually in termina, there might be errors not visable in Ul:
sudo systemctl stop dzga-flask #If service is running
cd /home/${USER}/DZGA-Flask
source env/bin/activate
python3 smarthome.py

This is the logging: `(env) pi@raspberrypi:~/DZGA-Flask $ python3 smarthome.py Smarthome server has started. Running without ssl

Serving Flask app 'smarthome'
Debug mode: on Smarthome server has started. Running without ssl Starting new HTTP connection (1): X.X.X.X:8084 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 Starting new HTTP connection (1): X.X.X.X:8084 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 Starting new HTTP connection (1): X.X.X.X:8084 Starting new HTTP connection (1): X.X.X.X:8084 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 Starting new HTTP connection (1): X.X.X.X:8084 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getdevices&plan=0&filter=all&used=true HTTP/1.1" 200 12707 Traceback (most recent call last): File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 2213, in call return self.wsgi_app(environ, start_response) File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/werkzeug/middleware/proxy_fix.py", line 182, in call return self.app(environ, start_response) File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 2193, in wsgi_app response = self.handle_exception(e) File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 2190, in wsgi_app response = self.full_dispatch_request() File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 1486, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 1484, in full_dispatch_request rv = self.dispatch_request() File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask/app.py", line 1469, in dispatch_request return self.ensure_sync(self.view_functions[rule.endpoint])(*view_args) File "/home/pi/DZGA-Flask/env/lib/python3.9/site-packages/flask_login/utils.py", line 290, in decorated_view return current_app.ensure_sync(func)(args, **kwargs) File "/home/pi/DZGA-Flask/modules/api.py", line 71, in gateway getDomoticzDevices(flask_login.current_user.username) File "/home/pi/DZGA-Flask/modules/domoticz.py", line 388, in getDomoticzDevices aog = getAog(d, user_id) File "/home/pi/DZGA-Flask/modules/domoticz.py", line 93, in getAog domain = getDomain(device) File "/home/pi/DZGA-Flask/modules/domoticz.py", line 35, in getDomain devs = devs.replace(" ", "") AttributeError: 'NoneType' object has no attribute 'replace' Starting new HTTP connection (1): X.X.X.X:8084 Starting new HTTP connection (1): X.X.X.X:8084 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getusers HTTP/1.1" 200 239 http://X.X.X.X:8084 "GET /json.htm?type=command&param=getversion HTTP/1.1" 200 331 `

DewGew commented 7 months ago

Try create a new user in DZGA-Flask

Xavier82 commented 7 months ago

Try create a new user in DZGA-Flask

I just did. I created new user with same credentials as user in Domoticz. Again with the same response. the devices_admin.json file hasn't got any devices :(

but still is my assumption correct that IF there is a connection between Domoticz and DZGA-flask (without google assistant enabled) that the devices overview should show me all devices from Domoticz?

Or does it only shows devices when Google Assistant is enabled?

Xavier82 commented 7 months ago

How is this possible?: error dzga-flask2

DewGew commented 7 months ago

flash mesages is not synced with whats acctually happens :) Shortcut in my coding ;)

DewGew commented 7 months ago

Try create a new user in DZGA-Flask

I just did. I created new user with same credentials as user in Domoticz. Again with the same response. the devices_admin.json file hasn't got any devices :(

but still is my assumption correct that IF there is a connection between Domoticz and DZGA-flask (without google assistant enabled) that the devices overview should show me all devices from Domoticz?

Or does it only shows devices when Google Assistant is enabled?

Even if Google Assitant is Disabled, the device should been showed.

Maybe if its devices that is not supported that makes the nonetype error. I usually create a user in domoticz then I add devices to that user just 2- 3 devices to start with.

Xavier82 commented 7 months ago

All kinds of devices of multiple different hardware are added to this user in Domoticz, so I could imagine that 1 device type or hardware might not be support, but I have over 6 different hardware and over 100 devices....

DewGew commented 6 months ago

Try create a seprate user for DZGA-Flask then add 1 switch to that user. Quite simple process.

edit: I got same issue if I get all my devices...I found the issue. Trying to rewrite code for this.

Xavier82 commented 6 months ago

ok, I did some tests. Before your rewrite of the code I created new user in Domoticz and added 4 devices, then did restart of server and then sync devices. In this case I get the 4 devices in DZGA-flask! The settings of the user in Domoticz are exactly the same as the user I used earlier.

I reconfigured back to the first user and did server restart and sync, no devices are shown, this user contains all devices in Domoticz.

After that I did a git pull to get the new code. Restart server and sync devices, still no devices.

DewGew commented 6 months ago

ok, I did some tests. Before your rewrite of the code I created new user in Domoticz and added 4 devices, then did restart of server and then sync devices. In this case I get the 4 devices in DZGA-flask! The settings of the user in Domoticz are exactly the same as the user I used earlier.

I reconfigured back to the first user and did server restart and sync, no devices are shown, this user contains all devices in Domoticz.

After that I did a git pull to get the new code. Restart server and sync devices, still no devices.

You are to fast .. now its merged

Xavier82 commented 6 months ago

haha, ok sorry about that, just did the update to 23.10 version, dispite still no solution, no devices from Domoticz, however there is a working connection to Domoticz since version is displayed....

UPDATE: on miraculous way it now works!!! Devices are now diplayed!

DewGew / DZGA-Flask

Domoticz devices not visible in DZGA-flask #62