Identifier rejected on 55U7KQ

marcandre83 commented 11 months ago

I bought a 55U7KQ that came out this year. It looks like Hisense did some changes to MQTT again, I am getting “Connection refused: Identifier rejected” over MQTT Explorer as well as within the mqtt logs on my raspberry although I am using my phone’s MAC address as the client ID. Also on this newer model I can not use the Remotenow app, only the VIDAA app. Any help would be highy appreciated.

barrel24 commented 11 months ago

i believe our issues my be linked however i cannot use the vidaa app only the remote now app. it is also only able to control volume now and wont do anything with a payload (changesource, launchapp etc) i am however abe to connect with mqtt explorer just check you have entered the certificates correctly for client cert and client key in mqtt explorer. other than that my knowledge is limited lol

LeoKlaus commented 11 months ago

Having the same issue on a 55U8KQ. I'm pretty sure Hisense changed the way MQTT auth works with newer models (those supporting only the Vidaa App).

I think this might be beyond the scope of what @Krazy998 is doing here, but here's what I could find so far:

When setting up and using the Vidaa App, I haven't been able to capture any MQTT traffic whatsoever using Wireshark. In general, there seems to be no direct communication between the TV and the device, as far as I could tell. The only packets the TV sent were DLNA and MDNS discovery related. Pretty much the same goes for the phone (tried an Android and iPhone).
I've looked through the files the Android app creates (/root/data/datacom.universal.remote.multi) and have found the following:
- There's a preferences file containing some identifiers for the device (the phone)
- There are two SQLite Databases: comm.db and mqttAndroidService.db, with the latter being empty in my case even after pairing and using the remote functionality.

`comm.db`contains a lot of information regarding the paired TV, with probably the most interesting table being `deviceconnectbean`. This table (in my case) contains the following data:	id	accesstoken	mac	refreshtoken	accesstoken_duration_day	accesstoken_time	refreshtoken_time	refreshtoken_duration_day
1	_+pUZJFGxZtnQsDYQEc3ewOv4dG/xJuvlR93qp7CVC6+pX/xN571Bh79/s76QWgS9	e0d8c4d2280088b8632f70f6	#+pUZJFGxZtnQsDYQEc3ewOv4dG/xJuvlR93qp7CVC6+juMFCy+y8Sv9I2PvnUysQ	2	1697805808	1697805808	30

This seems to indicate that the authentication process is much more complex now and I have no idea where to start with this. Anyways, I'll attach the database so you can have a look at them: comm.db.zip

Krazy998 commented 11 months ago

Thank you for sharing @LeoKlaus - I was going to suggest wireshark however now that hisense is using TLS it would have been difficult. But worth a initial try. I think its important to establish if the new Vidaa app actually is using a local connect to the tv or is it using some broker or service on the internet. Wireshark should at least tell us that hopefully.

The only other suggestion is to create a serial cable which can TTL directly to the TV and pull the config / app files for analysis. Im not sure how the new 55U8KQ TV's are, but my older 75" has a headphone jack (there is a service jack on the back of the tv) which effectively can connect to a usbTTL device.

I haven't done this in a while however - When the TV is booting you can interrupt its boot and get local access. I found I could only a small portion of the filesystem is set to read only. There is a script that is run as root that kicks off the update check where you can modify it to also launch a telnet server with root access. This will give you ability to pull the code off the device (if it hasnt been patched already on newer devices).

anonymous-one commented 11 months ago

Just want to chime in... I battled with this for quite some time before giving up.

I own a recently purchased (Oct 2023) 65E7KQ PRO which only connects via the VIDAA app.

I can pretty much repeat everything listed above. I ended up buying a broadlink IR blaster as all I cared about was ON / OFF.

I'll add another small (annoying) nugget : With this set, wifi / network connection is not established until after the TV has been powered on at least once, regardless of what you set in the menus etc. Pretty annoying for those of us who power socket the set OFF for the night (linked to my home alarm state etc...)...

anonymous-one commented 11 months ago

From what I recall when I was trying to overcome all of this after the initial handshake (eg: the TV was listed in the app), I was able to control it even when I firewalled it off from WAN access.

99% sure, but someone should verify, as I overcame most of what I wanted to do with a broadlink IR blaster and gave up...

Thank you for sharing @LeoKlaus - I was going to suggest wireshark however now that hisense is using TLS it would have been difficult. But worth a initial try. I think its important to establish if the new Vidaa app actually is using a local connect to the tv or is it using some broker or service on the internet. Wireshark should at least tell us that hopefully.

The only other suggestion is to create a serial cable which can TTL directly to the TV and pull the config / app files for analysis. Im not sure how the new 55U8KQ TV's are, but my older 75" has a headphone jack (there is a service jack on the back of the tv) which effectively can connect to a usbTTL device.

I haven't done this in a while however - When the TV is booting you can interrupt its boot and get local access. I found I could only a small portion of the filesystem is set to read only. There is a script that is run as root that kicks off the update check where you can modify it to also launch a telnet server with root access. This will give you ability to pull the code off the device (if it hasnt been patched already on newer devices).

LeoKlaus commented 11 months ago

The whole setup process and control does work without the TV having internet access.

I've tried capturing and decrypting the traffic from the app to the TV, but haven't been able to do so for lack of a rootable Android device. On both Android and iOS, the app ignores the system proxy configuration and uses certificate pinning, so a simple mitm doesn't work.

On Android, I've been able to at least capture the MQTT traffic (still on port 36669) using PCAPdroid.

The app itself uses Baidu protect. This (among other things) prevents you from running with USB debugging enabled.

If someone has a rooted Android device and a recent Hisense TV and wants to try getting this to work, the following might help: Anything that modifies the APK will prevent the app from starting, so your best bet is injecting code at runtime. https://codeshare.frida.re/@masbog/frida-android-unpinning-ssl/ IIRC, there are also Magisk modules that help bypassing certificate pinning.

If that works and the TV can still be controlled, you should be able to capture and decrypt the traffic using PCAPdroid with the MITM add-on (as far as I could tell, all the commands are still sent on port 36669).

It would be really cool to get this to work, but decompilation (and to some extent reverse engineering) is not exactly legal here in Germany, so even if I managed to break into the app, I couldn't post my findings.

chimpzilla commented 10 months ago

I have a brand new A6K 2023 model with the latest version of the Vidaa protocol. I've tried for a few weeks to solve this and had some progress. The APK cannot be reverse engineered properly with the likes of apktool etc. CFR had the best results, but still not very good. Frida just won't work at all with this app, it must have a lot of anti-frida code running and must be fairly up to date. You can view some of the apps debugging output though from USB debugging using ADB/Android Studio Logcat without doing anything to the apk. Just enable dev mode in Android and plug it into your PC with Android Studio's logcat running.

If you try a MITM (unpinning doesn't seem to work), the TV drops the connection. However, I was able to see what the initial connection message was including the username, password and client ID and I was able to connect for a limited amount of time using this information using the MQTT Explorer. Here is what I've found. Client ID: C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001 The first part of that is the mac address of the Android device. I was using Android 9, later versions don't allow access to this information, so they must generate a random ID. I don't know what the "44DE1F" is, but it doesn't seem to change, Username: his$1700688335 The number is UNIX time (number of seconds since 1970) Password: 10121AEA780F5D61B6BD93C202F0A6AE This is an encrypted value based on the client ID and the username. If I change any value in either the client name or the username, none of it works. It generates a fresh username based on the Unix time, this works for at least and hour (the TV must therefore be checking the time itself). I suspect that the password is encrypted using the private key located inside the p12 keystore. The reason I suspect this is because in the logcat the following information is displayed at the same time. 38D65DC30F45109A369A86FCE866A85B$C0:BD:D1:3D:6E:3E keyValue ==>41 value ==> E11C3113BA7285BBC64CAFA18CFB3055 keyNewpass == multiscreen123,userNewName = his$1700688335,passNewword === [C@d0eef3f topic: /mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001

The "multiscreen123" is the password for the keystore, so this is why I think this might be the case. However, it could be a key hardcoded into the java, which we can't see as it won't decompile. The "passNewword ===[C@d0eef3f" I guess is the "10121AEA780F5D61B6BD93C202F0A6AE", but they didn't convert the byte array to hex for debugging. Shame because it would make testing a lot easier than having to use a rooted phone with PCAPdroid

LeoKlaus commented 10 months ago

Very interesting findings. I'm really surprised you were able to use Logcat, as I remember seeing something along the lines of

if(debugger.isEnabled) {
    kill(self.pid)
}

(don't remember the exact implementation but it was very clear what it was supposed to do) in the decompiled code.

I didn't quite understand what information let you connect, though. ClientID and username is pretty self-explanatory from your comment (though I don't understand who in their right mind would make this depend on timestamps, of all things).

The password you used to connect was 10121AEA780F5D61B6BD93C202F0A6AE, as in that's the plaintext thing?

The key takeway (for me) here is that, even if somebody managed to reverse engineer or find the algorithm/key they use to determine the password, the whole timestamp thing would make it a giant pain in the ass to integrate this into a smart home system. I don't think node-red, ioBroker and the likes even support dynamic generation of login data, let alone using some self-defined algorithm including time stamps...

chimpzilla commented 10 months ago

Yes, that's the plain text password, hex values. Here's the full captured plaintext MQTT..C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001..his$1700688335..10121AEA780F5D61B6BD93C202F0A6AE

vs the older way MQTT..mqtt-explorer-96747f72..hisenseservice..multimqttservice

chimpzilla commented 10 months ago

The timestamp has been put in to stop it being just hardcoded and used on any TV. Hisense obviously don't want people to be able to control their own TV! If I gave anyone with an up to date TV a fresh set of login details I'm 99% sure that they would work if they used them within an hour. I don't think there's anything in there related to the TV in the details (I could be wrong) and the TV doesn't check the mac address because I managed to connect from my laptop that has a different mac address. I will check this at some point by checking with a Hisense TV running in a shop (I have one down the road from me).

chimpzilla commented 10 months ago

Looks like they work for about 4 hours in each direction. I just captured these details with my clock set to 4 hours time. client id: C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001 username: his$1700762502 password: 696CFC0716D0F4DAE4DF350E1BF3C725

I can confirm these are working with my Hisense. If anybody wants to try them in the next 8 hours or so just to prove that the TV details aren't involved. You'll notice that even once it's expired that the error in MQTT explorer is no longer "invalid client ID" but "Not authorised"

LeoKlaus commented 10 months ago

I keep getting "Disconnected from server" within MQTTExplorer. Tried your MAC, and both the "private" one and the real MAC of my iPhone.

This is really interesting though. If what you assume is true, this implementation does not only suck ass for us users but also from a security-standpoint.

chimpzilla commented 10 months ago

What certs do you have setup? I get that error message of the certificate is added,

LeoKlaus commented 10 months ago

What certs do you have setup? I get that error message of the certificate is added,

None, actually. I just got a new Mac and didn't carry over any of the MQTTExplorer settings. I just installed it.

chimpzilla commented 10 months ago

LeoKlaus commented 10 months ago

Okay, I've used the certs from d3nd3/Hisense-mqtt-keyfiles.

~Now, I get "Connection Refused: Not authorized" when using your MAC and "Identifier Rejected" when using any of mine.~ I'm an idiot and had the wrong password in the password field. Using your MAC and credentials, I can indeed connect to the broker.

chimpzilla commented 10 months ago

Excellent. Saves me having to get out of bed, go to my local TV shop and loiter around in the carpark with my laptop! We now know that there's no "extra" info put into the password or the client ID from the TV itself. If we can correctly generate the password to match the clientID and username we can connect to any TV/ It all revolves around the Unix timestamp to generate the password (the client ID is in there too, but this probably can be anything). The password is too short to be simply the clientID+username encrypted, it must be some sort of hash/hmac value related to what it would be when encrypted. If I alter any of the information in either the clientID or username then it fails Without knowing what format that takes, it's going to be very difficult to workout, but I will keep trying.

MaxwellJK commented 9 months ago

Hey @chimpzilla i also own the same model (A6K) and i would be happy to help as much as i can. I am just reading what you have found and it is amazing - thanks for it and for sharing. Have you made any progress? thanks

chimpzilla commented 9 months ago

No luck, sorry. It's pretty hardened to trying work out what's going on.

Krazy998 commented 9 months ago

I think only way would be to get root access to the tv via serial console (if that is still possible) and examine the code. Clearly hisense has gone out of its way to limit the ability to manage their latest TV's.

On a sidenote - I have root access to my old 3 year hisense and I have examined their logs and various other things. I can tell you they send logs to unified-ter-na.hismarttv.com which include things like what is playing on the tv (using data from the tv guide) what source is selected.

MaxwellJK commented 9 months ago

I have dug up a little bit more and by using logcat i found a lot of interesting details.

First of all, my token lasts 2 days instead of 4 hours. I didn't check if it was exactly 48 hours but in my logcat i have this entry "accesstoken_duration_day": 2, with the access token being created on 1702403040 epoch time (Tue Dec 12 2023 17:44:00 GMT).
Second, and probably more interesting, thing is that i was able to find a RefreshToken lasting 30 days and this log MQTT_CONNECT_INFO: url === ssl://192.168.1.234:36669,ip == 192.168.1.234 with something i assume are temporary username, access token and client id (then replaced with the official one mentioned above).
the app sends requests to these URLs https://vidaa-base-auth.vidaahub.com/aaa/get_dev_serial and https://vidaa-base-auth.vidaahub.com/aaa/sign_in - i wonder if the creds are actually generated by an external server and not by the tv.

There is much more in the logcat, just not sure it is useful. i'll keep digging

chimpzilla commented 9 months ago

I've found out a lot more information including how to generate the password for pairing. However I am stuck on the last step of pairing, hopefully somebody can help with this, as I've ran out of ideas. After pairing an access token is generated (looks like OAUTH maybe) and then you have to reconnect using this token as the password, you can then control the TV. All of this information is generated from the TV itself, nothing comes from a server on the internet. You can verify this yourself by switching off your internet connection, you can still pair with the TV using the Vidaa app and control it.

So this is how the password and client ID are generated, they are a series of MD5 hashes based on some fixed data, the current unix timestamp and the connecting device's mac address . You can use any mac address, as the TV does not verify this.

You can use this online tool to generate this info yourself: https://www.miraclesalad.com/webtools/md5.php (Make sure you switch on the "Treat multiple lines as separate strings" & "uppercase hashes") https://www.unixtimestamp.com/ (get a current unix timestamp) https://onlinetoolz.net/sum (add up the individual numbers of the time stamp)

These are the series of hashes you need to use: &vidaa#^app 38D65DC30F45109A369A86FCE866A85B$C0:BD:D1:3D:6E:3E his9h*i&s%e!r^v0i1c9 1701415028$3D5AEF

These are the results 38D65DC30F45109A369A86FCE866A85B 44DE1F1BC56E2737276E4D8F96E4AB53 3D5AEFF5F89E96E412ACF430C630DE9F 97A11ED3305C23F70B8A335F9D4C0CBF

How you get the hashes: The 1st line "&vidaa#^app" is fixed. The 2nd line is made up of "38D65DC30F45109A369A86FCE866A85B$" plus the mac address... "38D65DC30F45109A369A86FCE866A85B$" is fixed. The 3rd line is mostly fixed, apart from the digit after "his", in this example it's "his9" This is generated from the unixtime stamp, which in this example was "1701415028", to get 9 you have to add up all the digits 1+7+0+1+4+1+5+0+2+8=29, 9 is the last digit of this sum, 29. (This took me some time to work out as there's no code you can read from the app) The 4th line is the timestamp+$+ the first 6 digits of the 3rd hash "3D5AEF" So from this you now have your login details to connect via MQTT

username his$1701415028 (unix timestamp) client ID :C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001 (client ID 44DE1F is the first six digits of the 2nd hash) password :97A11ED3305C23F70B8A335F9D4C0CBF (4th hash)

Once you connect in using these login details, you send the following topics, subscribes. I couldn't get this to work in a windows based MQTT client (I tried MQTT Explorer and MQTTX), I found that it would connect, but that I didn't get any replies and when I subscribed the connection would close. I've not got much experience of MQTT though

I could get most of it to work by either sending the raw TCP data or using the Android Paho MQTT client (this is what the Vidaa app uses).

The topic contains the MQTT clientID

publish topic :/remoteapp/tv/ui_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/actions/vidaa_app_connect message: {"app_version":2,"connect_result":0,"device_type":"Mobile App"} Pin should show on screen.

subscribe: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/ui_service/data/authentication receivedmsg:

publish topic :/remoteapp/tv/ui_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/actions/authenticationcode message: {"authNum":"5299"} 5299 is the pin that is shown on the TV, when this is sent the pin should vanish from the screen.

subscribe: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/ui_service/data/authenticationcode receivedmsg: {"result":1,"info":""}

or if incorrect pin receivedmsg: {"result":100,"info":"Wrong authNum!!"}

publish topic:/remoteapp/tv/platform_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/data/gettoken message:{"refreshtoken":""}

publish topic:/remoteapp/tv/ui_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/actions/authenticationcodeclose message:

subscribe: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/platform_service/data/tokenissuance receivedmsg:{ "accesstoken": "_ZN5aYxg0DNMirIAXJ5OxWZ8mJaUcdR4GzkHXBofpx0a84hQAcbIaHhMxMhO/4hLf", "accesstoken_time": "1702028255", "accesstoken_duration_day": 2, "refreshtoken": "#ZN5aYxg0DNMirIAXJ5OxWZ8mJaUcdR4GzkHXBofpx0Z5nuorpAkQ9hUjZT8JfRRp", "refreshtoken_time": "1702028255", "refreshtoken_duration_day": 30 }

I can get all of this to work, including the "{"result":1,"info":""} message back from the TV, which confirms the pin is correct. But....... the TV never replies with the access token. I can see from the logcat from the Vidaa app that should happen. I've done a memory-dump of the Android phone and can see that this accesstoken reply does look like it was received from the TV via MQTT (just in case it was generated in some way in the vidaa app). The next step for me is to make up the USB lead and see what is going on in the TV, if I can, but I've been at this for some weeks now and need to do some other work (Although I hate being beaten). I've tried doing different orders for the subscribe messages and using different QoS settings. In the logcat the last few publish are sent within a couple of ms of each other. I think that the "codeclose" just removes the pin from the screen, although this goes anyway if you sent the correct pin. It's a shame the TV disconnects when doing a MITM attack, because then I could verify all the MQTT commands. If anyone else could try the above using a different client, that might be helpful as it could be some sort of additional setting. Someone had fun making this all so complicated, for no real reason!

LeoKlaus commented 9 months ago

You're an absolute madlad! I'll later try to reproduce this with my unit and see how it goes.

Someone had fun making this all so complicated, for no real reason!

It's insane. Hisense could've literally spent less time just providing some official MQTT integration/documentation and turned this utter shitshow into a great unique selling point for their hardware (be it for a very small target audience).

chimpzilla commented 9 months ago

Update Hilariously whilst just checking my instructions made sense and could be replicated, I just got the whole thing to work in MQTTX and the TV replied with the tokens. I didn't send any subscribes, the TV just replied. This never worked before when I tried using the MQTTX client, . I tried the login details in MQTT Explorer and got no reply. I'll try and figure out what is going on, but it confirms 100% that the TV responds with the access token via MQTT. I don't yet understand how the refresh token is used, but one step at a time.

Here's a walkthrough of what was sent and received via the Windows MQTTX client. Received messages in bold Topic: /remoteapp/tv/ui_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/actions/vidaa_app_connect {"app_version":2,"connect_result":0,"device_type":"Mobile App"} 2023-12-14 15:01:51:356

Topic: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/ui_service/data/authentication "" 2023-12-14 15:01:51:415

Topic: /remoteapp/tv/ui_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/actions/authenticationcode {"authNum":"2926"} 2023-12-14 15:02:07:548

Topic: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/ui_service/data/authenticationcode {"result":1,"info":""} 2023-12-14 15:02:07:587

Topic: /remoteapp/mobile/broadcast/ui_service/data/hotelmodechange {"hotel_mode":"off"} 2023-12-14 15:02:07:624

Topic: /remoteapp/tv/platform_service/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/data/gettoken {"refreshtoken":""} 2023-12-14 15:03:04:262

Topic: /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001/platform_service/data/tokenissuance { "accesstoken": "_7FNfOHs23fiiLEWdLrAvxdSqOu+sgPnlOGBJEKM/W3ZbJRG2GVi6TSW8OdCVYTvt", "accesstoken_time": "1702566185", "accesstoken_duration_day": 2, "refreshtoken": "#7FNfOHs23fiiLEWdLrAvxdSqOu+sgPnlOGBJEKM/W3aq/+fi3l3fgDDMxMPHKnWH", "refreshtoken_time": "1702566185", "refreshtoken_duration_day": 30}

chimpzilla commented 9 months ago

I just tried again in MQTTX and can't get it to respond. I've tried using fresh login details, but nothing comes back from the TV (the pin is displayed on the TV, so it's connected and the messages are sent correctly). Weird, weird, weird.

MaxwellJK commented 9 months ago

you are amazing @chimpzilla! ahh i also couldn't make MQTT Explorer work...i thought i was doing something wrong! I will try later tonight or tomorrow your procedure with MQTTX and let you know

chimpzilla commented 9 months ago

you are amazing @chimpzilla! ahh i also couldn't make MQTT Explorer work...i thought i was doing something wrong! I will try later tonight or tomorrow your procedure with MQTTX and let you know

Did MQTT Explorer connect?

MaxwellJK commented 9 months ago

Did MQTT Explorer connect?

it did yes but that was the only thing it was doing. no publishing, no subscribing. Tried with and without TLS (it was connecting in both cases).

chimpzilla commented 9 months ago

Did MQTT Explorer connect?

it did yes but that was the only thing it was doing. no publishing, no subscribing. Tried with and without TLS (it was connecting in both cases).

Mine won't connect unless it's TLS enabled. Here's some I just generated that will work for a few hours, if you want to test: username: his$1702583685 password: 2B9EA0238C4E90CB629D38E272664B68 clientID: C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001

If you post your connection details I can check they are valid?

MaxwellJK commented 9 months ago

I tried connecting now with your credentials and the connection keeps dropping in mqttx (although it connects). Yeah, more than happy to share my creds as soon as I understand how i can keep my connection stable

LeoKlaus commented 9 months ago

I managed to connect and get a token using MQTT Explorer on a Mac!

It wouldn't connect using the details I tried to create using your instructions. I ended up using the one you just posted.

I have a feeling that there are some pretty short timeouts for some of these operations. I'll try to spin up a python or bash script to automate the process.

chimpzilla commented 9 months ago

I managed to connect and get a token using MQTT Explorer on a Mac!

That's cool, I'll dig out my macbook tomorrow and give it a try with that.

It wouldn't connect using the details I tried to create using your instructions. I ended up using the one you just posted.

If you post the details and the hashes I'll check them over.

LeoKlaus commented 9 months ago

If you post the details and the hashes I'll check them over.

I must've made a mistake somewhere. I can now successfully connect using selfmade credentials.

I started writing a python script but won't have time to continue working on it until tomorrow or saturday.

Creating credentials and publishing already works (as in, triggers the TV to display the code and dismiss it), but I can't manage to receive any messages from the topics I subscribe to (or try to subscribe to). I am pretty clueless when it comes to MQTT, so maybe someone else can figure out what I'm missing. Anyway, here's the current code (forgive the messiness, it's a wip...) You'll have to have both the private key and certificate chain next to the script (line 44, for me its certfile="./rcm_certchain_pem.cer", keyfile="./rcm_pem_privkey.pkcs8") and change the IP in line 74 to the IP of your TV.

import re, uuid
import hashlib
import time
import paho.mqtt.client as mqtt

def cross_sum(n):
   r = 0
   while n:
       r, n = r + n % 10, n // 10
   return r

def stringToHash(input: str):
    result = hashlib.md5(input.encode("utf-8"))
    return result.hexdigest().upper()

timestamp = int(time.time())
#timestamp = 1702583685

firstHash = stringToHash("&vidaa#^app")

mac = ':'.join(re.findall('..', '%012x' % uuid.getnode())).upper()
#mac = "C0:BD:D1:3D:6E:3E"

secondHash = stringToHash("38D65DC30F45109A369A86FCE866A85B$" + mac)

lastDigitOfCrossSum = cross_sum(timestamp)%10

thirdHash = stringToHash("his"+ str(lastDigitOfCrossSum) +"h*i&s%e!r^v0i1c9")

fourthHash = stringToHash(str(timestamp) + "$" + thirdHash[:6])

#print(firstHash)
#print(secondHash)
#print(thirdHash)
#print(fourthHash)

clientID = mac + "$his$" + secondHash[:6] + "_vidaacommon_001"

client = mqtt.Client(client_id=clientID, clean_session=True, userdata=None, protocol=mqtt.MQTTv311, transport="tcp")

client.tls_set(ca_certs=None, certfile="./rcm_certchain_pem.cer", keyfile="./rcm_pem_privkey.pkcs8", cert_reqs=mqtt.ssl.CERT_NONE,
    tls_version=mqtt.ssl.PROTOCOL_TLS, ciphers=None)

client.tls_insecure_set(True)

username = "his$" + str(timestamp)

client.username_pw_set(username=username, password=fourthHash)

topicBasepath = "/remoteapp/tv/ui_service/" + clientID + "/"

def on_connect(client, userdata, flags, rc):
    print("Connected with result code "+str(rc))

    # Subscribing in on_connect() means that if we lose the connection and
    # reconnect then subscriptions will be renewed.
    client.subscribe("#")
    client.subscribe( topicBasepath + "ui_service/data/authentication")
    client.subscribe(topicBasepath + "ui_service/data/authenticationcode")
    client.subscribe( topicBasepath + "actions/authenticationcodeclose")
    client.subscribe( topicBasepath + "platform_service/data/tokenissuance")

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, msg):
    print(msg.topic+" "+str(msg.payload))

client.on_connect = on_connect
client.on_message = on_message

client.connect("192.168.27.140", 36669, 60)

client.loop_start()

client.publish( topicBasepath + "actions/vidaa_app_connect", '{"app_version":2,"connect_result":0,"device_type":"Mobile App"}')

client.subscribe( topicBasepath + "ui_service/data/authentication")

authNum = input("Enter the four digits displayed on your TV: ")

client.publish( topicBasepath + "actions/authenticationcode", '{"authNum":' + authNum + '}')

msg = client.subscribe(topicBasepath + "ui_service/data/authenticationcode")

#print(msg)
#if msg.payload == '{"result":1,"info":""}':
#print("Success! Getting access token...")
client.publish( topicBasepath + "data/gettoken", '{"refreshtoken":""}')
client.subscribe( topicBasepath + "actions/authenticationcodeclose")
#accessTokenResponse = client.subscribe( topicBasepath + "platform_service/data/tokenissuance")
#print("%s %s" % (msg.topic, msg.payload))
#else:
#print("%s %s" % (msg.topic, msg.payload))

#client.loop_forever()
input()
client.loop_stop()

chimpzilla commented 9 months ago

I tried with my macbook using MQTT explorer and got no responses. I managed to lockup the TV at one point today, requiring the power to go off at the power socket (standby doesn't properly reboot it), even the official app wouldn't connect. Great stuff with the Python, I have been working in Java/Android on my project. I'm thinking there must be some step missing that gets the TV to start responding. On the older non encrypted MQTT sets you have to make two subscribe requests regularly (10-20 seconds) for them to respond.

/remoteapp/mobile/broadcast/ /remoteapp/mobile/C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001 I've tried these with the 2023 model, but with no luck.

chimpzilla commented 9 months ago

There's a bit in the logcat that makes reference of the ClientID before anything else is sent AlarmPingSender com.universal.remote.multi D Register alarmreceiver to MqttServiceMqttService.pingSender.C0:BD:D1:3D:6E:3E$his$44DE1F_vidaacommon_001

"AlarmPingSender " is part of the Paho Android MQTT client I believe. I'm guess this might be what makes the TV respond.

MaxwellJK commented 9 months ago

nice stuff the python code - some of the topics were incorrect and i rewrote it also for my own benefit (first time dealing with MQTT but i used the one you wrote @LeoKlaus thanks for it).

Here's my python code - i literally just tested it and been able to create the tokens (and tested with success on my tv)

import re, uuid
import hashlib
import time
import paho.mqtt.client as mqtt
import json

reply = None

authentication_payload = None
authentication_code_payload = None
tokenissuance = None

def cross_sum(n):
   r = 0
   while n:
       r, n = r + n % 10, n // 10
   return r

def stringToHash(input: str):
    result = hashlib.md5(input.encode("utf-8"))
    return result.hexdigest().upper()

def on_connect(client, userdata, flags, rc):
    if rc == 0:
        client.connected_flag=True #set flag
        print("connected ok")

        # Subscribing in on_connect() means that if we lose the connection and
        # reconnect then subscriptions will be renewed.
        client.subscribe("#")

        client.subscribe(topicTVUIBasepath + "actions/vidaa_app_connect")
        client.subscribe(topicMobileBasepath + 'ui_service/data/authentication')
        client.subscribe(topicMobileBasepath + 'ui_service/data/authenticationcode')

        client.subscribe("/remoteapp/mobile/broadcast/ui_service/data/hotelmodechange")

        client.subscribe(topicMobileBasepath + 'platform_service/data/tokenissuance')

    else:
        print("Bad connection Returned code=",rc)
        client.bad_connection_flag=True

# The callback for when a PUBLISH message is received from the server.
def on_message(client, userdata, message):
    print("i'm on message")
    global reply
    print("message received " ,str(message.payload.decode("utf-8")))
    print("message topic=",message.topic)
    print("message qos=",message.qos)
    print("message retain flag=",message.retain)
    reply = message

def on_subscribe(client, userdata, mid, granted_qos):
    print("Subscribed: "+str(mid)+" "+str(granted_qos))

def on_publish(client, userdata, mid):
    print("Published message " + str(mid))

def on_disconnect(client, userdata, rc):
    print("disconnecting reason  "  +str(rc))

def on_log(client, userdata, level, buf):
    print("log: ",buf)

def on_message_msgs(mosq, obj, msg):
    print("MESSAGES: " + msg.topic + " " + str(msg.qos) + " " + str(msg.payload))

def on_authentication(mosq, obj, msg):
    global authentication_payload
    authentication_payload = msg

def on_authentication_code(mosq, obj, msg):
    global authentication_code_payload
    authentication_code_payload = msg

def on_tokenissuance(mosq, obj, msg):
    global tokenissuance
    tokenissuance = msg

timestamp = int(time.time())
#timestamp = 1702583685

firstHash = stringToHash("&vidaa#^app")

mac = ':'.join(re.findall('..', '%012x' % uuid.getnode())).upper()
#mac = "C0:BD:D1:3D:6E:3E"
print(f'mac {mac}')

secondHash = stringToHash("38D65DC30F45109A369A86FCE866A85B$" + mac)

lastDigitOfCrossSum = cross_sum(timestamp)%10

thirdHash = stringToHash("his"+ str(lastDigitOfCrossSum) +"h*i&s%e!r^v0i1c9")

fourthHash = stringToHash(str(timestamp) + "$" + thirdHash[:6])

print(firstHash)
print(secondHash)
print(thirdHash)
print(fourthHash)

clientID = mac + "$his$" + secondHash[:6] + "_vidaacommon_001"

client = mqtt.Client(client_id=clientID, clean_session=True, userdata=None, protocol=mqtt.MQTTv311, transport="tcp")

client.tls_set(ca_certs=None, certfile="./rcm_certchain_pem.cer", keyfile="./rcm_pem_privkey.pkcs8", cert_reqs=mqtt.ssl.CERT_NONE,
    tls_version=mqtt.ssl.PROTOCOL_TLS, ciphers=None)

client.on_connect = on_connect
client.on_message = on_message
# client.on_subscribe = on_subscribe
client.on_publish = on_publish
client.on_disconnect = on_disconnect
# client.on_log = on_log

client.connected_flag=False

client.tls_insecure_set(True)

username = "his$" + str(timestamp)
print(f'username: {username}')
print(f'password: {fourthHash}')
print(f'client_id: {clientID}')

client.username_pw_set(username=username, password=fourthHash)

topicTVUIBasepath = "/remoteapp/tv/ui_service/" + clientID + "/"
topicTVPlatformBasepath = "/remoteapp/tv/platform_service/" + clientID + "/"
topicMobileBasepath = "/remoteapp/mobile/" + clientID + "/"

client.message_callback_add(topicMobileBasepath + 'ui_service/data/authentication'          , on_authentication)
client.message_callback_add(topicMobileBasepath + 'ui_service/data/authenticationcode'      , on_authentication_code)

client.message_callback_add('/remoteapp/mobile/broadcast/ui_service/data/hotelmodechange'   , on_message_msgs)

client.message_callback_add(topicMobileBasepath + 'platform_service/data/tokenissuance'     , on_tokenissuance)

client.connect_async("192.168.1.234", 36669, 60)
client.loop_start()

while not client.connected_flag: #wait in loop
    print("In wait loop")
    time.sleep(1)

print('publishing message to actions/vidaa_app_connect ...')
client.publish( topicTVUIBasepath + "actions/vidaa_app_connect", '{"app_version":2,"connect_result":0,"device_type":"Mobile App"}')

print(f'subscribing to {topicMobileBasepath}ui_service/data/authentication ...')
while authentication_payload is None:
    time.sleep(0.1)

if authentication_payload.payload.decode() != '""' :
    print('Problems with the authentication message!')
    print(authentication_payload.payload)
    print('Exiting...')
    exit()

authNum = input("Enter the four digits displayed on your TV: ")

print(f'publishing message to {topicTVUIBasepath}actions/authenticationcode ...')
client.publish( topicTVUIBasepath + "actions/authenticationcode", '{"authNum":' + authNum + '}')

print(f'subscribing to {topicMobileBasepath}ui_service/data/authenticationcode ...')

client.subscribe(topicMobileBasepath + 'ui_service/data/authenticationcode')

while authentication_code_payload is None:
    time.sleep(0.1)

print(authentication_code_payload.payload.decode())
if json.loads(authentication_code_payload.payload.decode()) != json.loads('{"result": 1,"info": ""}') :
    print('Problems with the authentication message!')
    print(authentication_code_payload.payload)
    print('Exiting...')
    exit()

print("Success! Getting access token...")
print(f'publishing message to {topicTVPlatformBasepath}data/gettoken ...')
client.publish( topicTVPlatformBasepath + "data/gettoken", '{"refreshtoken": ""}')

print(f'publishing message to {topicTVUIBasepath}actions/authenticationcodeclose ...')
client.publish( topicTVUIBasepath + "actions/authenticationcodeclose")

print(f'subscribing to /remoteapp/mobile/broadcast/ui_service/data/hotelmodechange ...')
client.subscribe('/remoteapp/mobile/broadcast/ui_service/data/hotelmodechange')

print(f'subscribing to {topicMobileBasepath}platform_service/data/tokenissuance ...')
client.subscribe(topicMobileBasepath + 'platform_service/data/tokenissuance')

while tokenissuance is None:
    time.sleep(0.1)

print(tokenissuance.payload.decode())

print('token issued...well done!')

input()
client.loop_stop()
client.disconnect()

exit()

The connection details are

username: his$1702901342
password: 743E5D05E2413EF598FDA584F836B54B
client_id: 36:BD:7A:BA:29:16$his$9D286C_vidaacommon_001

The token instead is the following one:

{
        "accesstoken":  "_kE6HxbQWQYvKPGTyB90dtLk6IBfq5P5Y7b46TcgXucddsCjWgOo2cXCFjGYh7mBA",
        "accesstoken_time":     "1702901347",
        "accesstoken_duration_day":     2,
        "refreshtoken": "#kE6HxbQWQYvKPGTyB90dtLk6IBfq5P5Y7b46TcgXucfILk4EhZKq8Njb1BPdy/On",
        "refreshtoken_time":    "1702901347",
        "refreshtoken_duration_day":    30
}

LeoKlaus commented 9 months ago

Nice, thanks for updating the script @MaxwellJK! I've just tried using it, but never receive the token, the loop just waits endlessly (after publishing message 11). This is the output I end up with:

In wait loop
connected ok
publishing message to actions/vidaa_app_connect ...
subscribing to /remoteapp/mobile/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/ui_service/data/authentication ...
Published message 7
Enter the four digits displayed on your TV: 3239
publishing message to /remoteapp/tv/ui_service/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/actions/authenticationcode ...
subscribing to /remoteapp/mobile/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/ui_service/data/authenticationcode ...
Published message 8
{"result":1,"info":""}
Success! Getting access token...
publishing message to /remoteapp/tv/platform_service/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/data/gettoken ...
publishing message to /remoteapp/tv/ui_service/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/actions/authenticationcodeclose ...
subscribing to /remoteapp/mobile/broadcast/ui_service/data/hotelmodechange ...
subscribing to /remoteapp/mobile/96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001/platform_service/data/tokenissuance ...
Published message 10
Published message 11

Am I missing something obvious here? The auth code window on the TV disappears and it does recognize the authNum sent as legit, it just never seems to respond with the actual token. Seems similar to what @chimpzilla is running into.

I really wonder what is making the difference here. Funnily enough, I'm using the same machine I have used to (successfully) connect manually via MQTT Explorer. I can't manage to connect anymore, though.

MaxwellJK commented 9 months ago

not gonna lie, i spent all day yesterday without receiving the token. switched the tv off and tried today before sharing the code and it worked flawlessly. i tested it again right now and it worked (although i had to try twice since the fist time didn't show the AuthNum on screen)..

my only suggestion would be to switch it off and on and try again - not sure if the messages get stuck in some sort of limbo. one thing i noticed was that i was having problems with connecting to the tv because the vidaa app on my phone (not opened!) was causing connection issues so i uninstalled it...

LeoKlaus commented 9 months ago

I just entered a random made up mac address instead of using the one on my Macbook and it worked instantly, even multiple times in a row using the same mac address.

Maybe the TV adds the mac address to some sort of (in memory?) blacklist after a certain amount of failed connections?

The moment I switch back to the real mac address of the mac (man, this is confusing), it stops working.

Has anybody managed to figure out how to actually control the TV once you have an access token?

MaxwellJK commented 9 months ago

i partially did before starting focusing on my actual job.

username and client are the same, the password is the access token. I then tried to subscribe to the list of sources (/remoteapp/mobile/xxx/ui_service/data/sourcelist where xxx is the same you used in the other topics, e.g.96:02:3B:0A:C7:1B$his$E41EB3_vidaacommon_001) and published to /remoteapp/tv/ui_service/xxx/actions/sourcelist (xxx as above) and i got the list.

i even tried to start an app (e.g. neflix) - the tv turned black, probably starting the app, but no app was actually starting. i guess the payload is not the same as the one described in README.md

MaxwellJK commented 9 months ago

btw i was re-reading your message above

Maybe the TV adds the mac address to some sort of (in memory?) blacklist after a certain amount of failed connections?

Given all the steps we have to go through to get the correct password, i wouldn't be surprised if they added a blacklist...

LeoKlaus commented 9 months ago

I've worked through some of the example commands given in the Readme and most of them actually still work!

I've created a spreadsheet here: https://docs.google.com/spreadsheets/d/17PXKnCMhRZUQ08At_qu2EisuADa-zVtscTfWCrrehmE/edit?usp=sharing You should be able to just edit the sheet and update it if you find anything.

LeoKlaus commented 9 months ago

i even tried to start an app (e.g. neflix) - the tv turned black, probably starting the app, but no app was actually starting. i guess the payload is not the same as the one you described in README.md

For my TV, the only thing that's needed to start an app is its url. For Netflix it's just netflix.

Try publishing

{
    "url": "netflix"
}

to /remoteapp/tv/ui_service/C1:BE:D1:3D:6E:3E$his$7E1991_vidaacommon_001/actions/launchapp

I just tried Plex and it worked!

MaxwellJK commented 9 months ago

i tried as you suggested but it didn't work. I tried with the full json {"url":"youtube","isunInstalled":false,"name":"YouTube","from":"","storeType":98,"appId":"3","move":true,"isFav":true,"httpIcon":"data:image/png;base64,iconDownloadhttps://img.vidaahub.com/vidaa/2023/11/202311090858299321.png"} and worked. I then started removing key-value pairs one after the other and at some point the tv stopped responding...there is an anti flooding system too!!

MaxwellJK commented 9 months ago

Thanks for the spreadsheet @LeoKlaus :) much appreciated, i'll edit if I find something new :)

LeoKlaus commented 9 months ago

there is an anti flooding system too!!

Yikes...

What model do you have? It's interesting that it wouldn't work on yours. I assume we're all using the ROW model with Vidaa, not the US one with Google TV?

MaxwellJK commented 9 months ago

Yes, Vidaa one...i wish it was google tv! Model is A6K 50"

derdershat commented 9 months ago

Hi,

thank you that you are working on this!

I have an U8KQ and before on my U8QF I could also change settings like gamma values. This new Vidaa App is just a remote and not as good as the remotenow app where you can also access the picture settings.
Do you already tried to access them via mqtt?

MaxwellJK commented 9 months ago

Hi,

thank you that you are working on this!

I have an U8KQ and before on my U8QF I could also change settings like gamma values. This new Vidaa App is just a remote and not as good as the remotenow app where you can also access the picture settings. Do you already tried to access them via mqtt?

Do you know what the topic was?

LeoKlaus commented 9 months ago

I've just successfully refreshed my accesstoken using the refreshtoken!

To do so, I've connected to the TV using the username and clientID from a former successful connection (in this case his$1702984312 and C2:BE:D1:3D:6E:3E$his$DEC5E8_vidaacommon_001) and the refreshtoken as password.

Then I've just published

{
"refreshtoken": "#euiPdb79wKiMwM1ouXP+nPoGt9jTNZop5KKBbAevhk87PO0LU4Cj1OSsBGVlBSP/"
}

to /remoteapp/tv/platform_service/C2:BE:D1:3D:6E:3E$his$DEC5E8_vidaacommon_001/data/gettoken the response came from /remoteapp/mobile/C2:BE:D1:3D:6E:3E$his$DEC5E8_vidaacommon_001/platform_service/data/tokenissuance and looked just like the response you receive when first authenticating:

{
    "accesstoken":  "_YVA6VSKTy8b6vohQdBqMvkGRIggWPs4W7abPh7p0WsSY6RrMn1l6GrI0oahY4mxa",
    "accesstoken_time": "1702985357",
    "accesstoken_duration_day": 2,
    "refreshtoken": "#euiPdb79wKiMwM1ouXP+nPoGt9jTNZop5KKBbAevhk87PO0LU4Cj1OSsBGVlBSP/",
    "refreshtoken_time":    "1702984320",
    "refreshtoken_duration_day":    30
}

Note that the refreshtoken did not change. I wonder if it will change when refreshing shortly before it expires or if there's a hard 30 day cap per authentication.

Krazy998 / mqtt-hisensetv

Identifier rejected on 55U7KQ #14