eslavnov / pylips

Control Philips TVs (2015+) and Ambilight (+ Hue) through their reverse-engineered API (+ MQTT support!)
MIT License
342 stars 60 forks source link

Determining TV port #24

Closed lucasvdh closed 5 years ago

lucasvdh commented 5 years ago

Hi,

I've been using your wiki as a reference for creating an app that allows a Homey to control Philips TV's. It's been a great help and I might be able to shed some light on a few more features, but that's not what this is about.

In your readme you state that the main difference between regular Philips TV's and Philips Android TV's is the different port which they use. That seemed pretty clear and straightforward, but I've ran into some issues with the Jointspace api since.

Current situation

Currently I've got a user of my app who has a 2016 Philips Android TV which doesn't listen on port 1926 at all. It does however require digest authentication and the use of https for private endpoints via port 1925.

My own Philips Android TV is from 2018 and listens on both port 1925 and 1926. But in my case only the public endpoints such as /6/system and /6/notifychange are accessible via de 1925 port (both http and https work on this port which feels a bit strange). Sending a pair request to my TV requires the use of port 1926 and confirming the pair request also needs to done on port 1926.

Determining the port

My question; do you know of a way to determine which of these ports should be used to fetch the public /6/system endpoint and which port should be used for confirming the pairing process and accessing private endpoints?

eslavnov commented 5 years ago

Hey @lucasvdh,

Glad to hear that the reference is useful for you!

As far as your question goes: I am extremely surprised to hear about Android TVs using port 1925 - I've never encountered such models! Would you mind sharing the response from the /6/system endpoint of this particular TV? I will then compare it with the responses from other TVs I've collected and maybe I'll see some difference.

I could also imagine that you can scan the ports or just try sending a request to a different port if the first one fails, but it's kinda a hacky solution... I am also trying to reach somebody from Philips who is responsible for the API, so if it works out I will ask them this question.

eslavnov commented 5 years ago

My own Philips Android TV is from 2018 and listens on both port 1925 and 1926. But in my case only the public endpoints such as /6/system and /6/notifychange are accessible via de 1925 port (both http and https work on this port which feels a bit strange). Sending a pair request to my TV requires the use of port 1926 and confirming the pair request also needs to done on port 1926.

Yep, I have the same behavior with my TV. My theory was that old (non-Android) API was built on port 1925 (legacy from old jointspace API), and then at some point they've added support for Android version on port 1926 (probably so the API could support both versions without any breaking changes). That's also the reason why /6/system and /6/notifychange are accessible via the 1925 port both with and without SSL (it's for Android versions only). Then you access the 6/system endpoint on port 1925 (which works on all versions) and based on the response you use either port 1925 or 1926 for future requests. But yeah, Android TV working on port 1925 goes completely against this little theory of mine...

lucasvdh commented 5 years ago

Thanks for answering my questions! I've made a little bit of progress since I posted my question.

I think I might have been mistaken about the Android TV listening only on port 1925. I've managed to debug the issue a bit more and have reached the conclusion that the confirm pair request call is most likely the issue. So your theory holds up. The reason I though port 1926 wasn't being used on that specific TV was because the confirmation call of the pairing process got a connection timeout on port 1926 and a connection reset on port 1925. I assumed that port 1925 was the correct endpoint at least responded. But port 1926 appears to be the correct one for confirming the pairing process. It's a bit weird that I get a connection timeout though. I think there's an issue with the signing of the digest www-authenticate header. This might cause the TV to not accept the connection or something might actually fail in the Jointspace server on the TV resulting in no response.

What I've managed to figure out so far is that:

Digest auth Getting the digest auth working in NodeJS with my Philips TV was a nightmare. None of the existing digest clients worked and I eventually just decided to write my own. One of the issues was that the existing digest auth libraries used a nonce count value of 1 to sign the request but formatted it like 00000001 in the header. This resulted in a totally different signed response. Another issue I found was that the existing libraries didn't uppercase the http verb when signing the header. After fixing those two issue my TV finally accepted the signed response. But I have no idea how different models handle this.

Because I wrote this client myself I think its the most likely place to still contain errors or unhandled edge-cases. Or maybe the TV of that guy doesn't wanna pair because the verb has to be used as lowercase, who knows?

Maybe you could tell me about your experience with Philips TVs and digest auth. And maybe this is another question you could ask your contact at Philips if you ever manage to reach them.

eslavnov commented 5 years ago

Good to hear that my theory seems to hold up :)

I had the same problem with digest in node.js: for some reason none of the various libs I've tried worked for me. You can print the request headers in python (don't remember exactly how, but it's something trivial like 'request.headers') and then try to reverse-engineer them (since they are working). This could help: https://tools.ietf.org/html/rfc2617#section-3 Besides this, I am afraid I don't have any other advice since I didn't really spend much time investigating this issue.

I'll be closing the issue since it's not pylips-related anymore, but good luck with it!

lucasvdh commented 5 years ago

Yes, no problem. Thanks for your help so far!

If you ever get in touch with the person responsible for the Jointspace api, let me know. It would be great if they just shared their api doc :smirk:.

lucasvdh commented 5 years ago

@eslavnov I think this issue should be reopened because I've found a case in which an Android TV with Jointspace v6.2.0 only listens on port 1925. So sadly it seems that your theory doesn't hold up.

Did you manage to get in touch with the person at Philips responsible for the Jointspace api? If you did or are still trying to, could you maybe also ask the following question:

"Are there more cases like the 49pus7101/12 model, in which Android Philips TV’s with Jointspace v6 only accept connections on port 1925? If so can these be identified by anything in the system response?"

Some info on the TV

The model of the TV is 49pus7101/12 and the system response is included below.

The owner of the TV mentioned that when trying to fetch the system response on port 1926 he got a "connection reset" with http as well as https.

The TV only returned a response when using http and port 1925.

System response:

{
    "notifyChange": "http",
    "menulanguage": "Dutch",
    "name": "tv-philips",
    "country": "Netherlands",
    "serialnumber_encrypted": "D+q4bijYOwT4hdDyS75aE73tMzZaSSzBQ9yiFMPqSTo=\n",
    "softwareversion_encrypted": "CzwJ+H33uwQTm2mfEteSJiSB+z0wRvl8WSwZu1itwlk=\n",
    "model_encrypted": "XUPZXApQvnWGsKtrK2EMVHfKchsmqySAkw268KA09Fk=\n",
    "deviceid_encrypted": "ZBu4F127jW6wQcc9Jg/CdmI29w12Oq152oSiDUX1/bU=\n",
    "nettvversion": "8.1.3",
    "epgsource": "one",
    "api_version": {
        "Major": 6,
        "Minor": 2,
        "Patch": 0
    },
    "featuring": {
        "jsonfeatures": {
            "editfavorites": [
                "TVChannels",
                "SatChannels"
            ],
            "recordings": [
                "List",
                "Schedule",
                "Manage"
            ],
            "ambilight": [
                "LoungeLight",
                "Hue",
                "Ambilight"
            ],
            "menuitems": [
                "Setup_Menu"
            ],
            "textentry": [
                "context_based",
                "initial_string_available",
                "editor_info_available"
            ],
            "applications": [
                "TV_Apps",
                "TV_Games",
                "TV_Settings"
            ],
            "pointer": [
                "not_available"
            ],
            "inputkey": [
                "key"
            ],
            "activities": [
                "intent"
            ],
            "channels": [
                "preset_string"
            ],
            "mappings": [
                "server_mapping"
            ]
        },
        "systemfeatures": {
            "tvtype": "consumer",
            "content": [
                "dmr",
                "dms_tad"
            ],
            "tvsearch": "intent",
            "pairing_type": "digest_auth_pairing",
            "secured_transport": "true"
        }
    }
}
eslavnov commented 5 years ago

Hey, thanks for the info! Do you know if this TV also uses port 1925 for pairing and other endpoints? Or is this purely about the 'system' endpoint? If it's only the 'system' endpoint, then I might have an idea...

I've contacted several people in both Philips and TP-Vision and even got some replies, but ultimately I was unable to reach the ones responsible for the API. I will probably try again in the future...

lucasvdh commented 5 years ago

Shame that you couldn't get in touch with a person responsible for the API, you'd expect there'd be someone to talk to with a technology that's in every Philips TV.

Regarding your question: I'm not sure if its only the system endpoint that only listens on port 1925.

I wrote a test script which the owner of this TV used to check some stuff for me. As far as I can see, when using port 1925 the system response is fetched and then the pairing process is started by calling pair/request (also on port 1925) which immediately receives a "connection reset". Maybe plain http needs to be used here? That would contradict the "secured_transport": "true" in the system response though.

I might be able to let him test if he is able to start the pairing on port 1926. But I think he tested this already and just got a timeout on port 1926. I'll try starting the pairing process via https, http and on both ports, you never know..

lucasvdh commented 5 years ago

Got some feedback regarding the 49pus7101/12 TV.

The pair/request endpoint for starting the pairing process is accessible, but only on port 1926 with https. A little bit of progress, so yay, I guess.

However when trying to confirm the pair by posting a signed request to pair/grant on port 1926 with https, it immediately gets a "connection reset". I might try some different port/https variations here as well, but I don't think that that will work. Got any speculations on possible solution?

eslavnov commented 5 years ago

I think the best course of action would be to ask the person to install Charles and then pair with the TV using the official Philips remote app while running Charles. Send me the logs from Charles and we'll know for sure what's going on and which ports/protocols/endpoints their TV uses.

(Charles is super-easy to use and they have a free trial)

eslavnov commented 5 years ago

The pair/request endpoint for starting the pairing process is accessible, but only on port 1926 with https. A little bit of progress, so yay, I guess.

Thanks, that's actually great news meaning that all Android TVs use port 1926 for pairing, so my theory still holds up :) Weird that the system endpoint does not respond on 1926, but not a big deal since its available on both versions on 1925 and we can just look for "pairing_type": "digest_auth_pairing" to check if it's the Android version.

lucasvdh commented 5 years ago

Yeah, that would be the easiest. Though I think this might be a bit above the expertise level of the owner of the TV.

I think this issue can closed again, seeing as how we now have a concrete answer to determining what port should be used for what endpoint. If I manage to find out something else about the pairing process I'll post it here as well.

Cheers