newAM / hisensetv

Python API to control Hisense brand TVs via their internal MQTT broker.
MIT License
37 stars 16 forks source link

Cannot get to authorise #14

Open i00 opened 3 years ago

i00 commented 3 years ago

I get the following when I try to authorise:

bash-5.0# hisensetv 10.4.1.158 --authorize
Traceback (most recent call last):
  File "/usr/local/bin/hisensetv", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/hisensetv/__main__.py", line 108, in main
    with tv:
  File "/usr/local/lib/python3.8/site-packages/hisensetv/__init__.py", line 117, in __enter__
    self._mqtt_client.connect(self.hostname, self.port)
  File "/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 941, in connect
    return self.reconnect()
  File "/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 1104, in reconnect
    sock.do_handshake()
  File "/usr/local/lib/python3.8/ssl.py", line 1309, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1125)

This is for the Hisense 100" L5F, and the Android app works fine.

newAM commented 3 years ago

Have you tried --no-ssl? Some models do not use SSL.

i00 commented 3 years ago

Yes, I did try that and it doesn't work either:

bash-5.0# hisensetv 10.4.1.158 --authorize --no-ssl
Traceback (most recent call last):
  File "/usr/local/bin/hisensetv", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/hisensetv/__main__.py", line 108, in main
    with tv:
  File "/usr/local/lib/python3.8/site-packages/hisensetv/__init__.py", line 125, in __enter__
    raise HisenseTvTimeoutError(f"failed to connect in {self.timeout:.3f}s")
hisensetv.HisenseTvTimeoutError: failed to connect in 10.000s
newAM commented 3 years ago

Nothing is jumping out to me as an obvious problem unfortunately.

Are you able to get a packet capture (using something like wireshark) when the android app connects to the TV?

i00 commented 3 years ago

Will try to do this within the next week but may not get the chance unfortunately as I will be busy over much of the Easter break

chinedu40 commented 3 years ago

Just to say I'm having the exact same issue as i00 using the 100 L5F Could you walk me through how to use wireshark to get a packet capture...I'm fairly computer literate but never used it app before.

newAM commented 3 years ago

To start out with wireshark just run it on your local computer, and access an HTTP (no S) site, make sure you can see the HTTP requests in wireshark. The process of filtering will be the similar when you're looking for MQTT packets from your phone app.


Now, unfortunately, when you want to packet capture from another device getting it setup is specific to your network.

For my network I have: phone <---> wireless access point <---> switch <---> computer

To get a packet capture I enable port mirroring on my switch to mirror packets from the WAP port to the port my PC is on (these are physical Ethernet ports in this context, not to be confused with socket ports).

Depending on your networking appliance(s) setting up port mirroring can be anything from a button in the web interface to an iptables edit. Also note that in my case I have a separate switch, most at-home setups have an all-in-one wireless access point / switch / router, so you will likely be looking for this in your router settings.

chinedu40 commented 3 years ago

Just to add something that might be helpful, looking at the forums i was able to use MQTT explorer and access my TV by following these instructions listed here https://github.com/d3nd3/Hisense-mqtt-keyfiles but i do not know if such a thing could be implemented into this application.

newAM commented 3 years ago

You can pass an SSL context into the constructor:

    def __init__(
        self,
        hostname: str,
        *,
        port: int = 36669,
        username: str = "hisenseservice",
        password: str = "multimqttservice",
        timeout: Union[int, float] = 10.0,
        enable_client_logger: bool = False,
        ssl_context: Optional[ssl.SSLContext] = None,
    ):

By default it is disabled, None, the certs should be bypassed entirely, other people have reported it working, but unfortunately my TV does not have certs so I cannot test this.

chinedu40 commented 3 years ago

I'm not sure how to pass the certs into the constructor, do you have any idea - i'm more than happy to test code.

newAM commented 3 years ago

The API is a bit unweidly since it is security focused, but you're looking for create_default_context.

You'll have to provide the paths as an arugment to that function:

cafile, capath, cadata represent optional CA certificates to trust for certificate verification

something like this

import ssl
from hisensetv import HisenseTv:

# replace None as needed
ssl_context = ssl.create_default_context(cafile=None, capath=None, cadata=None)

with HisenseTv("your_hostname", ssl_context=ssl_context):
    pass

print("it worked")
chinedu40 commented 3 years ago

Thanks for the kickstart newAM. So i am getting a slightly different error message now using the following added code in main.py

if args.no_ssl: ssl_context = None else: ssl_context = ssl.create_default_context(cafile= None, capath=None, cadata=CA_ROOT_PEM) -->> CA_ROOT_PEM is defined at the beginning of main.py

Traceback (most recent call last): File "hisensetv.py", line 6, in main.main() File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/hisensetv/main.py", line 196, in main with tv: File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/hisensetv/init.py", line 138, in enter self._mqtt_client.connect(self.hostname, self.port) File "/var/packages/py3k/target/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 941, in connect return self.reconnect() File "/var/packages/py3k/target/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 1104, in reconnect sock.do_handshake() File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1108)

I'm unsure how to proceed from here. I've tried to disable certificate verification but that just leads to the original error.

if args.no_ssl: ssl_context = None else: ssl_context = ssl.create_default_context(cafile= None, capath=None, cadata=CA_ROOT_PEM) ssl_context.check_hostname = False ssl_context.verify_mode = ssl.CERT_NONE

Traceback (most recent call last): File "hisensetv.py", line 6, in main.main() File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/hisensetv/main.py", line 196, in main with tv: File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/hisensetv/init.py", line 138, in enter self._mqtt_client.connect(self.hostname, self.port) File "/var/packages/py3k/target/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 941, in connect return self.reconnect() File "/var/packages/py3k/target/usr/local/lib/python3.8/site-packages/paho/mqtt/client.py", line 1104, in reconnect sock.do_handshake() File "/volume1/homes/chinedu40/homebridge-hisense-tv-remotenow/bin/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1108)

Any further ideas would bee greatly appreciated

chinedu40 commented 3 years ago

Okay, i finally was able to get it to authorise using the keys from here https://github.com/d3nd3/Hisense-mqtt-keyfiles.

please see the edited main.py file below to allow this to work for this particular model 100L5.

!/usr/bin/env python3.8

import argparse import json import logging import ssl from . import HisenseTv from hisensetv import HisenseTv

def main(): parser = argparse.ArgumentParser(description="Hisense TV control.") parser.add_argument("hostname", type=str, help="Hostname or IP for the TV.") parser.add_argument( "--authorize", action="store_true", help="Authorize this API to access the TV.", ) parser.add_argument( "--ifname", type=str, help="Name of the network interface to use", default="" ) parser.add_argument( "--get", action="append", default=[], choices=["sources", "volume", "state"], help="Gets a value from the TV.", ) parser.add_argument( "--key", action="append", default=[], choices=[ "power", "up", "down", "left", "right", "menu", "back", "exit", "ok", "volume_up", "volume_down", "channel_up", "channel_down", "fast_forward", "rewind", "stop", "play", "pause", "mute", "home", "subtitle", "netflix", "youtube", "amazon", "0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "source_0", "source_1", "source_2", "source_3", "source_4", "source_5", "source_6", "source_7", ], help="Sends a keypress to the TV.", ) parser.add_argument( "--no-ssl", action="store_true", help="Do not connect with SSL (required for some models).", ) parser.add_argument( "-v", "--verbose", action="count", default=0, help="Logging verbosity." )

args = parser.parse_args()

if args.verbose:
    level = logging.DEBUG
else:
    level = logging.INFO

root_logger = logging.getLogger()
stream_handler = logging.StreamHandler()
formatter = logging.Formatter(
    fmt="[{asctime}] [{levelname:<8}] {message}", style="{"
)
stream_handler.setFormatter(formatter)
root_logger.addHandler(stream_handler)
root_logger.setLevel(level)
logger = logging.getLogger(__name__)

if args.no_ssl:
    ssl_context = None
else:
    ssl_context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
    ssl_context.load_cert_chain(certfile='/path/to/rcm_certchain_pem.cer', keyfile='/path/to/rcm_pem_privkey.pkcs8')

tv = HisenseTv(
    args.hostname, enable_client_logger=args.verbose >= 2, ssl_context=ssl_context, network_interface=args.ifname
)
with tv:
    if args.authorize:
        tv.start_authorization()
        code = input("Please enter the 4-digit code: ")
        tv.send_authorization_code(code)

    for key in args.key:
        func = getattr(tv, f"send_key_{key}")
        logger.info(f"sending keypress: {key}")
        func()

    for getter in args.get:
        func = getattr(tv, f"get_{getter}")
        output = func()
        if isinstance(output, dict) or isinstance(output, list):
            output = json.dumps(output, indent=4)
        print(output)

if name == "main": main()

newAM commented 3 years ago

Okay, i finally was able to get it to authorise using the keys from here

Excellent! There is a bug in the code then :(. The certs should be bypassed (and one person confirmed it worked?).


Is this bit in the constructor relevant at all to the functionality? network_interface=args.ifname


If you are able to run more experiments can you try this?

    if args.no_ssl:
        ssl_context = None
    else:
        ssl_context = ssl._create_unverified_context(purpose=ssl.Purpose.CLIENT_AUTH)
chinedu40 commented 3 years ago

Okay, i finally was able to get it to authorise using the keys from here

Excellent! There is a bug in the code then :(. The certs should be bypassed (and one person confirmed it worked?).

Is this bit in the constructor relevant at all to the functionality? network_interface=args.ifname

If you are able to run more experiments can you try this?

    if args.no_ssl:
        ssl_context = None
    else:
        ssl_context = ssl._create_unverified_context(purpose=ssl.Purpose.CLIENT_AUTH)

Regarding network_interface=args.ifname this does not affect functionality in fact the code is from a person who's made a homebridge plugin based on your code. See here https://github.com/MrAsterisco/homebridge-hisense-tv

so when i used:

if args.no_ssl:
    ssl_context = None
else:
    ssl_context = ssl._create_unverified_context(purpose=ssl.Purpose.CLIENT_AUTH)

i get the same error originally namely: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/bin/hisensetv", line 8, in sys.exit(main()) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/hisensetv/main.py", line 108, in main with tv: File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/hisensetv/init.py", line 117, in enter self._mqtt_client.connect(self.hostname, self.port) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/paho/mqtt/client.py", line 941, in connect return self.reconnect() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/paho/mqtt/client.py", line 1104, in reconnect sock.do_handshake() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:1123)

if i do if args.no_ssl: ssl_context = None else: ssl_context = ssl._create_unverified_context(purpose=ssl.Purpose.CLIENT_AUTH) ssl_context.load_cert_chain(certfile='/path/to/rcm_certchain_pem.cer', keyfile='/path/to/rcm_pem_privkey.pkcs8')

it allows me to authorize with my device. I think the --no-ssl argument is working as intended as even with the above if i include --no-ssl it won't allow me to authorise ands timeout.

Happy to test more code if needed!

chinedu40 commented 3 years ago

Having spent hours getting the homebridge plugin that utilises your code to work, i would mark this as closed because i think this is a workaround for models that require these files to connect. Perhaps making reference to it in the README file would suffice?

newAM commented 3 years ago

I will leave the issue open since this should be fixed, implementing SSL and using the certs right answer, but I do not have the time or the correct TV to verify the code above is secure. Adding the code as-is would be step above the current implementation of attempting to bypass certs, but I do not want to provide a false sense of security.

ryanshand commented 3 years ago

@chinedu40 thanks for your notes on this, I've read through your changes and spent a little bit of time on this, but to be honest I'm a HOOBS N00B ;-)

I am running a Raspberry Pi VM on my laptop with HOOBS 3.3.5 running. It has integrated into my Aircon and alarm system without issue, so I'm confident I've got the basics right.

I have a Hisense 70S5 running version V0000.01.00T.L0305 software. I can connect to it using RemoteNOW app on my iPhone.

When I installed this plugin (https://github.com/MrAsterisco/homebridge-hisense-tv) I initially got the same mqtt errors you got until I installed paho on my python3.7 release. I now get the below error as you reported:

4/27/2021, 4:32:37 PM Starting to advertise 'HOOBSS 7310' using bonjour-hap backend! 4/27/2021, 4:32:37 PM [HiSenseTV] An error occurred while fetching inputs: TypeError: Cannot read property 'join' of null

I downloaded MQTT Explorer on my PC and following the steps you outlined above, using the certificates, I can browse the TV endpoint successfully (I didn't need to use an Android device).

I edited the main.py to match what you noted above, copied the certificates to /home/pi/Hisensecerts/ directory and restarted the HOOBS service.

However I still get the same error - I'm not sure if what I'm doing is dumb (it probably is) but it doesn't seem like swapping the .py files actually worked at all - not sure if they're pulled into some cache or memory, but i cleared the __pycache folder as well and that didn't help.

May I ask how you managed to test this? or if you have any recommendation on how I can replace the file so it is picked up by the service?

thanks

Ryan

main .zip

chinedu40 commented 3 years ago

@chinedu40 thanks for your notes on this, I've read through your changes and spent a little bit of time on this, but to be honest I'm a HOOBS

May I ask how you managed to test this? or if you have any recommendation on how I can replace the file so it is picked up by the service?

i used terminal to test my implementation as it gives more verbose logs. Can you paste the output of the following command python3.X hisensetv.py —(double dash)authorise IPADDRESS You will have to download the files from this repo and change the main.py

ryanshand commented 3 years ago

working.zip I actually just got it working right now, using the attached files I had to modify a bit.

I can now from command line run python3.7 hisensetv.py 192.168.1.32 --authorize --ifname eth0 and I was prompted for the code, and it came up on the TV.

So that is great progress thanks a lot!

Now I am just not sure how to get these new python files into my plugin. I guess I will research this more.

thanks,

gjlamb commented 2 years ago

Having same issues with this and another installation on a new U9G tv. I managed to get mqtt explorer to connect using key files from https://github.com/d3nd3/Hisense-mqtt-keyfiles.

Is there a way to use this app with certs?

TV appears to be listening.

hoobs@hoobs:~ $ nmap -p 36000-37000 192.168.0.42 Starting Nmap 7.70 ( https://nmap.org ) at 2022-01-02 17:03 AEDT Nmap scan report for 192.168.0.42 Host is up (0.0028s latency). Not shown: 999 filtered ports PORT STATE SERVICE 36669/tcp open unknown 36670/tcp open unknown

Nmap done: 1 IP address (1 host up) scanned in 4.18 seconds

Downloaded both pairs of new files from repo and working.zip above but getting errors.

hoobs@hoobs:/usr/local/lib $ hisensetv 192.168.0.42 --authorize Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 5, in from hisensetv.main import main File "/usr/local/lib/python3.8/site-packages/hisensetv/init.py", line 7 <!DOCTYPE html>

Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 5, in from hisensetv.main import main ImportError: cannot import name 'main' from 'hisensetv.main' (/usr/local/lib/python3.8/site-packages/hisensetv/main.py)

newAM commented 2 years ago

certs are ignored by default, you can pass a custom SSL context if desired.

As for that error it looks like an install error, not a connection error, I'm not exactly sure what is going on there unfortunately.

ryanshand commented 2 years ago

Having same issues with this and another installation on a new U9G tv. I managed to get mqtt explorer to connect using key files from https://github.com/d3nd3/Hisense-mqtt-keyfiles.

Is there a way to use this app with certs?

TV appears to be listening.

hoobs@hoobs:~ $ nmap -p 36000-37000 192.168.0.42 Starting Nmap 7.70 ( https://nmap.org ) at 2022-01-02 17:03 AEDT Nmap scan report for 192.168.0.42 Host is up (0.0028s latency). Not shown: 999 filtered ports PORT STATE SERVICE 36669/tcp open unknown 36670/tcp open unknown

Nmap done: 1 IP address (1 host up) scanned in 4.18 seconds

Downloaded both pairs of new files from repo and working.zip above but getting errors.

hoobs@hoobs:/usr/local/lib $ hisensetv 192.168.0.42 --authorize Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 5, in from hisensetv.main import main File "/usr/local/lib/python3.8/site-packages/hisensetv/init.py", line 7

Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 5, in from hisensetv.main import main ImportError: cannot import name 'main' from 'hisensetv.main' (/usr/local/lib/python3.8/site-packages/hisensetv/main.py)

hey mate, if you're using the main.py and init.py from the "working.zip" file above, then ssl is enabled by default and will be looking in /home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/ for the 2 cert files.

the error "ImportError: cannot import name 'main' from 'hisensetv.main' (/usr/local/lib/python3.8/site-packages/hisensetv/main.py)" is generally due to either a circular import (two files trying to import eachother) or you have more than one .py file with a main() function in the same folder.

If you've put these files in the same directory as the default HiSenseTV plugin files that might cause this problem, I run these from my /home directory on my Pi

gjlamb commented 2 years ago

/home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/

does that directory supposed to exist already? Doesn’t exist on my install

ryanshand commented 2 years ago

Nah that’s just on my box. You can change it to match wherever you want to put them. Just update the path

keeping in mind that my zip file was a POC and those changes are already merged into the main branch.

ryanshand commented 2 years ago

@gjlamb have a read if this thread it may help

https://github.com/MrAsterisco/homebridge-hisense-tv/issues/3

gjlamb commented 2 years ago

@gjlamb have a read if this thread it may help

MrAsterisco/homebridge-hisense-tv#3

Thanks. That’s the one I started working with but haven’t been able to get it going either

gjlamb commented 2 years ago

I thinks its all above my pay grade :).

Copied files from working.zip to /usr/local/lib/python3.8/site-packages/hisensetv/. Edited main.py with locations and names of cer and pkcs8 files. Still get error below.

hoobs@hoobs:/tmp/working $ hisensetv 192.168.0.42 --authorize Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 8, in sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/hisensetv/main.py", line 108, in main ssl_context.load_cert_chain(certfile='/home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/cert.cer', ssl.SSLError: [SSL] PEM lib (_ssl.c:4022)

ryanshand commented 2 years ago

The error SSLError: [SSL] PEM lib (_ssl indicates the path to the PEM file could not be found.

check your path and filenames in main.py

gjlamb commented 2 years ago

The error SSLError: [SSL] PEM lib (_ssl indicates the path to the PEM file could not be found.

check your path and filenames in main.py

Looks ok... (this site seems to drop when posting?)

hoobs@hoobs:/usr/local $ hisensetv 192.168.0.42 --authorize Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 8, in sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/hisensetv/main.py", line 108, in main ssl_context.load_cert_chain(certfile='/home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/cert.cer', ssl.SSLError: [SSL] PEM lib (_ssl.c:4022) hoobs@hoobs:/usr/local $ cat /usr/local/lib/python3.8/site-packages/hisensetv/main.py | grep cer ssl_context.load_cert_chain(certfile='/home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/cert.cer', keyfile='/home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/key.pkcs8') hoobs@hoobs:/usr/local $ ls /home/hoobs/.hoobs/node_modules/homebridge-hisense-tv-remotenow/Hisensecerts/ cert.cer key.pkcs8 rcm_certchain_pem.cer rcm_pem_privkey.pkcs8

gjlamb commented 2 years ago

Also tried the unverified SSL context from above with same error

if args.no_ssl:
 ssl_context = None
else:
    ssl_context = ssl._create_unverified_context(purpose=ssl.Purpose.CLIENT_AUTH)
    ssl_context.load_cert_chain(certfile='/tmp/cert.cer', keyfile='/tmp/key.pkcs8')

hoobs@hoobs:/tmp $ hisensetv 192.168.0.42 --authorize --ifname wlan0 Traceback (most recent call last): File "/usr/local/bin/hisensetv", line 8, in sys.exit(main()) File "/usr/local/lib/python3.8/site-packages/hisensetv/main.py", line 111, in main ssl_context.load_cert_chain(certfile='/tmp/cert.cer', keyfile='/tmp/key.pkcs8') ssl.SSLError: [SSL] PEM lib (_ssl.c:4022)

gjlamb commented 2 years ago

As mentioned above, I'm able to connect to the TV using MQTT explorer with key_files certs from https://github.com/d3nd3/Hisense-mqtt-keyfiles. Not sure if looking at that repo would give insight on getting it working?

gjlamb commented 2 years ago

I notice when using MQTT-Explorer that I need to disable validating certificates. Is there an option to do this with the script?