App does not send cookie in api request

[breenstorm]

Describe the bug Since one of the last updates, both lite and premium are unable to connect. 'Oops' page appears saying credentials are invalid. No settings on both the app and the server were changed. Credentials are verified and correct and were working just fine before the update.

To Reproduce Open app, error appears.

Expected behavior App should connect.

Screenshots Screenshot

Smartphone (please complete the following information):

• Device: Xiaomi Mi A3
• OS: Android 10
• Domoticz system version 3.8153
• Domoticz app version Latest

Additional context I ran a tcpdump on the server side and can see the auth is successful but the resulting SID cookie is not sent in the request right next after auth, so Domoticz replies http 403.

Here is the dump (with credentials removed);

09:07:25.381475 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [S], seq 1697957158, win 65535, options [mss 1460,sackOK,TS val 113268071 ecr 0,nop,wscale 9], length 0 E..<WI@.@... B.. B.).,.Pe4.&................... ..Ug.......
09:07:25.381636 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [S.], seq 3943424986, ack 1697957159, win 28960, options [mss 1460,sackOK,TS val 250988 ecr 113268071,nop,wscale 7], length 0 E..<..@.@.$G B.) B...P.,....e4.'..q ........... ...l..Ug.... 09:07:25.386832 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [.], ack 1, win 172, options [nop,nop,TS val 113268308 ecr 250988], length 0 E..4WJ@.@... B.. B.).,.Pe4.'.........W..... ..VT...l 09:07:25.387161 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [P.], seq 1:267, ack 1, win 172, options [nop,nop,TS val 113268309 ecr 250988], length 266: HTTP: GET /json.htm?type=command&param=logincheck&username=&password= HTTP/1.1 E..>WK@.@... B.. B.).,.Pe4.'........V...... ..VU...lGET /json.htm?type=command&param=logincheck&username=&password= HTTP/1.1 User-Agent: Dalvik/2.1.0 (Linux; U; Android 10; Mi A3 Build/QKQ1.190910.002) Host: 10.66.0.41 Connection: Keep-Alive Accept-Encoding: gzip

09:07:25.387294 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [.], ack 267, win 235, options [nop,nop,TS val 250988 ecr 113268309], length 0 E..47.@.@... B.) B...P.,....e4.1........... ...l..VU 09:07:25.394663 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [P.], seq 1:511, ack 267, win 235, options [nop,nop,TS val 250989 ecr 113268309], length 510: HTTP: HTTP/1.1 200 OK E..27.@.@... B.) B...P.,....e4.1........... ...m..VUHTTP/1.1 200 OK Content-Length: 102 Content-Type: application/json;charset=UTF-8 Cache-Control: no-cache Pragma: no-cache Access-Control-Allow-Origin: * Content-Encoding: gzip Set-Cookie: SID=d6a32ae550b15831c07f11f0a39e1969_NTA2OGRhZTItZDkzNS00MzZhLTk4YWItZDA5ZTQzODk1ODM3.1588922245; HttpOnly; path=/; Expires=Fri, 08 May 2020 07:17:25 GMT Connection: Keep-Alive Keep-Alive: max=20, timeout=20

............RPPP.L.()VR.R0.....$........ b%.%9.`..........l.Tiqj.X&).. V.ZT.....6..045V.........p... 09:07:25.396458 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [.], ack 511, win 174, options [nop,nop,TS val 113268318 ecr 250989], length 0 E..4WL@.@... B.. B.).,.Pe4.1........|B..... ..V^...m 09:07:26.147946 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [P.], seq 267:465, ack 511, win 174, options [nop,nop,TS val 113268953 ecr 250989], length 198: HTTP: GET /json.htm?type=command&param=getconfig HTTP/1.1 E...WN@.@..: B.. B.).,.Pe4.1............... ..X....mGET /json.htm?type=command&param=getconfig HTTP/1.1 User-Agent: Dalvik/2.1.0 (Linux; U; Android 10; Mi A3 Build/QKQ1.190910.002) Host: 10.66.0.41 Connection: Keep-Alive Accept-Encoding: gzip

09:07:26.148969 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [P.], seq 511:667, ack 465, win 243, options [nop,nop,TS val 251064 ecr 113268953], length 156: HTTP: HTTP/1.1 403 Forbidden E...7.@.@... B.) B...P.,....e4......RC..... ......X.HTTP/1.1 403 Forbidden Content-Length: 85 Content-Type: text/html

403 Forbidden

09:07:26.154893 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [P.], seq 465:731, ack 667, win 174, options [nop,nop,TS val 113269076 ecr 251064], length 266: HTTP: GET /json.htm?type=command&param=logincheck&username=&password= HTTP/1.1 E..>WO@.@... B.. B.).,.Pe4.....u....N...... ..YT....GET /json.htm?type=command&param=logincheck&username=&password= HTTP/1.1 User-Agent: Dalvik/2.1.0 (Linux; U; Android 10; Mi A3 Build/QKQ1.190910.002) Host: 10.66.0.41 Connection: Keep-Alive Accept-Encoding: gzip

09:07:26.168066 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [P.], seq 667:1177, ack 731, win 252, options [nop,nop,TS val 251066 ecr 113269076], length 510: HTTP: HTTP/1.1 200 OK E..27.@.@... B.) B...P.,...ue4............ ......YTHTTP/1.1 200 OK Content-Length: 102 Content-Type: application/json;charset=UTF-8 Cache-Control: no-cache Pragma: no-cache Access-Control-Allow-Origin: * Content-Encoding: gzip Set-Cookie: SID=d5d88129d36bf8197b2d688dd47e0e5b_NWYzY2Q4MGMtNGFjOC00MTBmLWI3ZDctZDU5M2E4YmQ4YTZj.1588922246; HttpOnly; path=/; Expires=Fri, 08 May 2020 07:17:26 GMT Connection: Keep-Alive Keep-Alive: max=20, timeout=20

............RPPP.L.()VR.R0.....$........ b%.%9.`..........l.Tiqj.X&).. V.ZT.....6..045V.........p... 09:07:26.176916 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [P.], seq 731:929, ack 1177, win 176, options [nop,nop,TS val 113269098 ecr 251066], length 198: HTTP: GET /json.htm?type=command&param=getconfig HTTP/1.1 E...WP@.@..8 B.. B.).,.Pe4.....s........... ..Yj....GET /json.htm?type=command&param=getconfig HTTP/1.1 User-Agent: Dalvik/2.1.0 (Linux; U; Android 10; Mi A3 Build/QKQ1.190910.002) Host: 10.66.0.41 Connection: Keep-Alive Accept-Encoding: gzip

09:07:26.177944 IP 10.66.0.41.http > 10.66.1.201.46380: Flags [P.], seq 1177:1333, ack 929, win 260, options [nop,nop,TS val 251067 ecr 113269098], length 156: HTTP: HTTP/1.1 403 Forbidden E...7.@.@... B.) B...P.,...se4......M4..... ......YjHTTP/1.1 403 Forbidden Content-Length: 85 Content-Type: text/html

403 Forbidden

09:07:26.211441 IP 10.66.1.201.46380 > 10.66.0.41.http: Flags [.], ack 1333, win 176, options [nop,nop,TS val 113269133 ecr 251067], length 0 E..4WQ@.@... B.. B.).,.Pe4..........r...... ..Y.....

[galadril]

Same issues as the pinned issue; #582

[breenstorm]

There is no mention of the api returning 403 in that issue, mainly things regarding basic auth (which I do not use, I use login page) and behaviour on invalid credentials.

[galadril]

if you look at the api calls that domoticz does after login, you'll probably see that web is doing the next login call:

GET /json.htm?type=command&param=logincheck&username=%username%&password=%password%

But this logincheck is deprecated... so you should update your system.

More info on this also on the forum: https://www.domoticz.com/forum/viewtopic.php?f=37&t=8884&p=246711#p246711

[breenstorm]

Look at my carefully constructed capture. The app is doing a GET for the login call with the username and password in the url (the 'really old' method).

If this is deprecated but the app still seems to use the deprecated version anyway, how can it be the cause of the next call (GET /json.htm?type=command&param=getconfig) returning 403? No matter what call, there should be some kind of token or session and the app is clearly not referencing any of them in the call after login.

[galadril]

Ok let me run some test

[breenstorm]

If I can make a suggestion; choose on what version to support and put minimum required version of Domoticz in your releasenotes on Google Play. Maybe even do a check before login with a notice your domoticz is outdated.

I am very satisfied with this app so far (which is why I paid for the premium, while I'm not using any of the benefits premium offers) and being a developer myself I truly understand the wonky codebase and undocumented major changes in the api of domoticz are a real pain in the ***, but if you are clear about what to expect (which version supported f.i.) then I wouldn't have bothered you knowing it is me who failed to keep my software up to date (and I could have acted on this by disabling updating for your app as long as I didn't upgrade my domoticz).

You may also know that the wonky codebase and the bugs every new version of domoticz comes with is the major reason for people not updating, at least for me ;-)

Good luck on finding the issue and let me know if there is anything I can help with.

[galadril]

Thats a good suggestion, but I only have 2 test systems available to test.. so most of the times I just can't know the minimal compatibility version.

But if you compare it with for example Philips Hue.. they just tell you in the app to update your base station.. they also don't have that much backwards compatibility. maybe a couple of versions... but they have proper versioning in their api;s.. which domoticz doesn't have...

so its a pain 🗡️

Could you maybe make an users for me that can toggle one dummy device on your system and send the credentials to domoticz@hnogames.nl ?

[breenstorm]

You have mail. Sorry for the delay, I was battling some of Domoticz's quircks.

[galadril]

New version is coming v0.2.217

[breenstorm]

Confirming it is fixed in the latest version.

[galadril]

Sweet