[FEAT] Available for testing

[mtzro2003]

First, I'd like to thank you for this! Beautiful! :+1:

I have a 49PUS8303 and I'm available for testing any new feature :smile: Looking forward to MQTT integration also - definitely big progress for home automation...

[eslavnov]

Hey @mtzro2003, thank you for your kind words! I was not sure if anybody was actually using this script, so I haven't really worked on Pylips recently... (since it already solves my use cases as it is)

So yeah, haven't really realized that people are waiting for updates, will try to do something about MQTT support in the coming week or two... I have a working prototype, just need to polish it a bit.

[gb160]

Hey @mtzro2003, thank you for your kind words! I was not sure if anybody was actually using this script, so I haven't really worked on Pylips recently... (since it already solves my use cases as it is)

So yeah, haven't really realized that people are waiting for updates, will try to do something about MQTT support in the coming week or two... I have a working prototype, just need to polish it a bit.

Hi, forgive my ignorance but will MQTT support have any bearing on input selection (hdmi) or lack of?

[eslavnov]

@gb160, no, the input selection is not possible with the API - there is nothing I can do as even the official app does not have this functionality. You can do it with ADB, although it is not super reliable in my experience (there is a link somewhere in the beginning of the Readme).

[gb160]

@eslavnov Thats a shame as unfortunately my TV is not Android, its just Linux.

Ive managed to achieve input switching via CEC commands, but its messy as hell...its a real shame the API has this function missing. Cheers for the reply anyway.

[eslavnov]

@mtzro2003, I've pushed a new version with some new commands and MQTT support - would you mind giving it a try? It is basically rewritten from scratch, so I hope I did not break anything, lol

[mtzro2003]

First thing tomorrow morning when I arrive @office. :D Gonna report back!

On Sun, Apr 21, 2019, 22:11 Evgeny Slavnov notifications@github.com wrote:

@mtzro2003 https://github.com/mtzro2003, I've pushed a new version with some new commands and MQTT support - would you mind giving it a try? It is basically rewritten from scratch, so I hope I did not break anything, lol

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eslavnov/pylips/issues/9#issuecomment-485275291, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQWPIQ6AEKW6W37CAQIK63PRS36BANCNFSM4HEHULBA .

[mtzro2003]

Sorry for not being able to report sooner... Here's what I did so far after upgrading. As I used pylips from command line with user and password, I put those in settings.ini and tried to run pylips - TV was OFF - actually OFF, not standby:

$ cat settings.ini
[DEFAULT]
verbose = True
MQTT_listen = False
MQTT_update = False
num_retries = 3
update_interval = 3
[TV]
host = 192.168.200.33
port =
apiv = 
user = XXXXXXXXXXXXX
pass = XXXXXXXXXXXXX
protocol =
[MQTT]
host = 127.0.0.1
port = 1883
user = pylips
pass = XXXXXXXXXXXXXX
TLS = False
topic_pylips =
topic_status =
$  python3 pylips
Checking API version and port...
Trying http://192.168.200.33:1925/6/system
Connection refused
Trying http://192.168.200.33:1925/5/system
Connection refused
Trying http://192.168.200.33:1925/1/system
Connection refused
PING 192.168.200.33 (192.168.200.33) 56(84) bytes of data.

--- 192.168.200.33 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

IP 192.168.200.33 seems to be offline. Exiting...

Question: shouldn't it try to ping first and THEN try to connect to API?

Second, TV ON:

$ python3 pylips.py
Checking API version and port...
Trying http://192.168.200.33:1925/6/system
Traceback (most recent call last):
  File "pylips.py", line 412, in <module>
    pylips = Pylips("settings.ini")
  File "pylips.py", line 72, in __init__
    if self.check_paired() is False:
AttributeError: 'Pylips' object has no attribute 'check_paired'

On line 72, I believe it should be if self.check_if_paired() is False I added apiv = 6 to settings.ini and it worked:

$ python3 pylips.py
Please enable mqtt_listen in settings.ini or provide a valid command with a '--command' argument

Gonna test it further with MQTT and get back to you

[mtzro2003]

I turned OFF the TV (standby actually), by remote, and pylips reports that it's ON. Here's the JSON that is posted to the MQTT topic:

{"power_on": true, "volume": 0, "muted": true, "cur_app": "org.droidtv.playtv", "ambilight": {"styleName": "FOLLOW_VIDEO", "isExpert": false, "menuSetting": "COMFORT", "stringValue": "Comfort"}, "ambihue": false}

Getting the power state from command line reply:

$ python3 pylips.py --host 192.168.200.33 --user XXXXXXXX --pass XXXXXXXXXXXXXXXXXXXX --command powerstate
Checking API version and port...
Trying http://192.168.200.33:1925/6/system
Sending GET request to https://192.168.200.33:1926/6/powerstate
Request sent!
{"powerstate":"Standby"}

Also, when TV is in standby pylips continues to interrogate the API, querying ambilight config, acivities and audio level. Maybe, when it detects that the TV is in standby or OFF, it should query only the powerstate.

After a while, it reports that the powerstate changed - TV was responding to ping:

{"power_on": false, "volume": null, "muted": false, "cur_app": "NA", "ambilight": null, "ambihue": false}

Another issue: If apiv parameter in settings.ini is configured manually, pylips gets no reply from the TV. When I leave it blank, everything is OK:

$ cat settings.ini
[DEFAULT]
verbose = True
MQTT_listen = True
MQTT_update = True
num_retries = 3
update_interval = 3
[TV]
host = 192.168.200.33
port = 1925
apiv = 6
user = XXXXXXXXXX
pass = XXXXXXXXXX
protocol = https://
[MQTT]
host = 127.0.0.1
port = 1883
user = pylips
pass = XXXXXXXXXX
TLS = False
topic_pylips = pylips/Livingroom_TV/command
topic_status = pylips/Livingroom_TV/status
openhab@openhab:~/pylips$ python3 pylips.py
Started MQTT status updater
Sending GET request to https://192.168.200.33:1925/6/audio/volume
Connected to MQTT broker at 127.0.0.1
Sending GET request to https://192.168.200.33:1925/6/audio/volume
Sending GET request to https://192.168.200.33:1925/6/audio/volume
Sending GET request to https://192.168.200.33:1925/6/activities/current
Sending GET request to https://192.168.200.33:1925/6/activities/current
Sending GET request to https://192.168.200.33:1925/6/activities/current
Sending GET request to https://192.168.200.33:1925/6/ambilight/currentconfiguration
Sending GET request to https://192.168.200.33:1925/6/ambilight/currentconfiguration
Sending GET request to https://192.168.200.33:1925/6/ambilight/currentconfiguration
Sending POST request to https://192.168.200.33:1925/6/menuitems/settings/current
Sending POST request to https://192.168.200.33:1925/6/menuitems/settings/current
Sending POST request to https://192.168.200.33:1925/6/menuitems/settings/current
^CTraceback (most recent call last):
  File "pylips.py", line 412, in <module>
    pylips = Pylips("settings.ini")
  File "pylips.py", line 95, in __init__
    self.start_mqtt_updater(self.verbose)
  File "pylips.py", line 409, in start_mqtt_updater
    time.sleep(int(self.config["DEFAULT"]["update_interval"]))
KeyboardInterrupt
openhab@openhab:~/pylips$ nano settings.ini
openhab@openhab:~/pylips$ cat settings.ini
[DEFAULT]
verbose = True
MQTT_listen = True
MQTT_update = True
num_retries = 3
update_interval = 3
[TV]
host = 192.168.200.33
port = 1925
apiv =
user = XXXXXXXXXX
pass = XXXXXXXXXX
protocol = https://
[MQTT]
host = 127.0.0.1
port = 1883
user = pylips
pass = XXXXXXXXXX
TLS = False
topic_pylips = pylips/Livingroom_TV/command
topic_status = pylips/Livingroom_TV/status
openhab@openhab:~/pylips$ python3 pylips.py
Checking API version and port...
Trying http://192.168.200.33:1925/6/system
Started MQTT status updater
Connected to MQTT broker at 127.0.0.1
Sending GET request to https://192.168.200.33:1926/6/audio/volume
Request sent!
{"muted":false,"current":1,"min":0,"max":60}
Sending GET request to https://192.168.200.33:1926/6/activities/current
Request sent!
{"component":{"packageName":"NA","className":"NA"}}
Sending GET request to https://192.168.200.33:1926/6/activities/tv
Request sent!
{"channel":{"ccid":1141,"preset":"117","name":"PRO TV HD"},"channelList":{"id":"allcab","version":"8"}}
Sending GET request to https://192.168.200.33:1926/6/ambilight/currentconfiguration
Request sent!
{"styleName":"FOLLOW_VIDEO","isExpert":false,"menuSetting":"COMFORT","stringValue":"Comfort"}
Sending POST request to https://192.168.200.33:1926/6/menuitems/settings/current
Request sent!
{"values":[{"value":{"Nodeid":2131230774,"Controllable":true,"Available":true,"string_id":"Switch on","data":{"value":false}}}],"version":0}
^CTraceback (most recent call last):
  File "pylips.py", line 412, in <module>
    pylips = Pylips("settings.ini")
  File "pylips.py", line 95, in __init__
    self.start_mqtt_updater(self.verbose)
  File "pylips.py", line 409, in start_mqtt_updater
    time.sleep(int(self.config["DEFAULT"]["update_interval"]))
KeyboardInterrupt

[eslavnov]

Hey @mtzro2003 , thank you for your input - this is extremely useful!

Question: shouldn't it try to ping first and THEN try to connect to API?

Well yes, but actually no :) PING sometimes takes quite a lot of time to complete, so I am using it purely as a safety net to help me debugging possible problems. So the logic is "Try the endpoints => if no response, then try to ping => let user know what has happened (e.g., if TV is responding to PING, but not the endpoints, then it's probably an API version issue...). So it's a feature, not a bug - although I do admit it's somewhat counter-intuitive. Maybe I should look into PING parameters and set a short timeout or something, I dunno. I'll think about it.

On line 72, I believe it should be if self.check_if_paired() is False

Yep, my bad!

I turned OFF the TV (standby actually), by remote, and pylips reports that it's ON. Here's the JSON that is posted to the MQTT topic:

Great, my TV behaves differently in Standby mode... Sometimes I really hate Phillips...

Maybe, when it detects that the TV is in standby or OFF, it should query only the powerstate.

Yeah, I think this is a good idea given how your TV handles standby mode.

After a while, it reports that the powerstate changed - TV was responding to ping:

Sorry, not sure what you mean here. TV was responding to ping, but Pylips was reporting it as OFF?

If apiv parameter in settings.ini is configured manually, pylips gets no reply from the TV. When I leave it blank, everything is OK:

Thanks, there is a small bug in port discovery (1925 vs 1926), should be easy to fix.

I should be able to make all the fixes today or tomorrow, I'll let you know when it's ready.

[mtzro2003]

After a while, it reports that the powerstate changed - TV was responding to ping:

Sorry, not sure what you mean here. TV was responding to ping, but Pylips was reporting it as OFF?

Here's how my TV, Philips 49PUS8303, behaves when you turn it "OFF" bye remote - OFF is actually a standby, something like when you turn off the screen on an Android phone or tablet: when I press the OFF button, the screen (of course) turns off, but the TV continues to respond to ping and to API interrogations. after some time (something like 5 minutes), the TV stops responding to ping and to API

What I meant was that pylips (in MQTT mode) reported the powerstate as ON even is the TV was turned "OFF" by remote. Querying directly the powerstate from command line (pylips old-style) reported that the TV is in standby. Not sure how to explain further. I'll wait for the new version and I'll fiddle with it some more...

[eslavnov]

Got it, thanks! Mine behaves the same with the exception that my "Standby" mode lasts for 5-10 seconds, then it becomes "OFF" and the API stops responding, so I haven't really implemented a proper "Standby" mode. That's super-easy to fix, will do.

Thank you again for your help, it's extremely hard to catch all the edge cases with just one TV model...

[eslavnov]

@mtzro2003 Quick question:

If apiv parameter in settings.ini is configured manually, pylips gets no reply from the TV. When I leave it blank, everything is OK:

So I've noticed that in the first scenario (where you had specified apiv 6) the port is 1925 in the config, that's why it's failing. In the second scenario (with blank apiv) the port is being overriden by apiv discovery, that's why it works.

So the question is: did you manually specify port 1925 in config (then it is working as intended) or was port 1925 added at some point by Pylips (then it's a bug)? Thanks!

[mtzro2003]

Damn! Indeed, there is a typo in settings.ini... My bad! I put the port manually and it was the wrong one... The port should be 1926.

On Thu, Apr 25, 2019 at 11:55 AM Evgeny Slavnov notifications@github.com wrote:

@mtzro2003 https://github.com/mtzro2003 Quick question:

If apiv parameter in settings.ini is configured manually, pylips gets no reply from the TV. When I leave it blank, everything is OK:

So I've noticed that in the first scenario (where you had specified apiv 6) the port is 1925 in the config, that's why it's failing. In the second scenario (with blank apiv) the port is being overriden by apiv discovery, that's why it works.

So the question is: did you manually specify port 1925 in config (then it is working as intended) or was port 1925 added at some point by Pylips (then it's a bug). Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eslavnov/pylips/issues/9#issuecomment-486580187, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQWPIWJJVZFIO6X62BPJFLPSFWY3ANCNFSM4HEHULBA .

[eslavnov]

Nice, one less bug to fix! :) Thanks for a quick response, appreciate it!

[eslavnov]

I've pushed an update, check it out when you have time! Thanks!

[mtzro2003]

Well, it's running smoothly for 1 day - no issues so far. Didn't yet had time to integrate it with openHAB - just 2-3 status items: powerstate, volume and such. I'll be out of town for the next week so, when I'll get back, I'll dig into it some more. Wonderful job, @eslavnov, thousand thanks!

[eslavnov]

Good to hear! Thank you again for your help with the testing!

[mtzro2003]

In the mean time... How would I use MQTT to retrieve the ambilight brightness for example... From what I understand, I should post to topic_pylips (in my case pylips/Livingroom_TV/command) a JSON like:

{ "command": "post", "path": "menuitems/settings/current", "body": {"nodes":[{"nodeid":2131230769}]} }

where 2131230769 is the menu entry for ambilight brightness. But how do I get the return value (JSON) in such a use case?

Also, it looks like I can't get it to work from command line, not MQTT - see below curl is working, but pylips is not, username and password are the same. What am I missing? (pylips is not running in MQTT mode in the example)

$ cat settings.ini
[DEFAULT]
verbose = True
MQTT_listen = True
MQTT_update = True
num_retries = 3
update_interval = 3
[TV]
host = 192.168.200.33
port = 1926
apiv = 6
user = XXXXXXXXX
pass = XXXXXXXXXX
protocol = https://
[MQTT]
host = 127.0.0.1
port = 1883
user = pylips
pass = XXXXXXXXXX
TLS = False
topic_pylips = pylips/Livingroom_TV/command
topic_status = pylips/Livingroom_TV/status

$ python3 pylips.py --command post --path menuitems/settings/current --body '{"nodes":[{"nodeid":2131230769}]}'
Sending POST request to https://192.168.200.33:1926/6/menuitems/settings/current
Request sent!
<html>
<head>
   <title>Status page</title>
</head>
<body style="font-family: sans-serif;">
<p style="font-size: 1.2em;font-weight: bold;margin: 1em 0px;">Bad Request</p>
<p>The request could not be understood by the server due to malformed syntax</p>
<p>You can get technical details <a href="http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.1">here</a>.<br>
Please continue your visit at our <a href="/">home page</a>.
</p>
</body>
</html>

$ curl -sS -X POST --digest --insecure -u XXXXXXXX:XXXXXXXXXXXX -d '{"nodes":[{"nodeid":2131230769}]}' https://192.168.200.33:1926/6/menuitems/settings/current
{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":2}}}],"version":0}

One more thing: maybe it would be useful to increase the verbosity... For example, to log MQTT messages from subscribed topic and published messaged to the other one...

[eslavnov]

Hey, so there was a small bug that affected some of the POST requests, I've just pushed a fix and updated README with better examples (see MQTT section), let me know if this helps with your issue. I've also noticed that you are sending different body when using pylips vs curl, so this might also be the reason....

For example, to log MQTT messages from subscribed topic and published messaged to the other one...

You can just subscribe to the MQTT topics from any client and see what's being published, so I am not sure there is much sense in logging it in Pylips...

[mtzro2003]

let me know if this helps with your issue.

Yes, now it's working - same command as above:

$ python3 pylips.py --command post --path menuitems/settings/current --body '{"nodes":[{"nodeid":2131230769}]}'
Sending POST request to https://192.168.200.33:1926/6/menuitems/settings/current
Request sent!
{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":2}}}],"version":0}

I've also noticed that you are sending different body when using pylips vs curl

What is different? I don't see any difference between them... But nevermind, now pylips is working :+1:

Moving forward and trying one of your examples: to change brightness through MQTT. Running

$ mosquitto_pub -h 127.0.0.1 -u pylips -P XXXXXXXXXXXXX -t pylips/Livingroom_TV/command -m '{"command":"post", "path": "menuitems/settings/update", "body": {"values":[{"value":{"Nodeid":2131230769,"Controllable":"true", "Available":"true","string_id":"Brightness", "data":{"value":10}}}]}}'

I saw somewhere on pylips' output a Please provide a 'path' argument error. Taking a look at the code I noticed the following lines:

                    if message["command"] == "get":
                        if len(path)>0:
                            return print("Please provide a 'path' argument")
                        self.get(path,self.verbose)
                    elif message["command"] == "post":
                        if len(path)>0:
                            return print("Please provide a 'path' argument")

changing the len(path)>0 to len(path) == 0 got it working :smile:

Trying mosquitto_pub -h 127.0.0.1 -u pylips -P XXXXXXXXXXXX -t pylips/Livingroom_TV/command -m '{"command":"ambilight_brightness", "body":{"value":10}}' doesn't work, but my python knowledge is limited... python3 pylips.py --command ambilight_brightness --body '{"value":10}' works, but sending the command to MQTT topic doesn't

updated README with better examples (see MQTT section)

Well, the examples are always welcome, but my question was about GETTING the value of ambilight brightness. Something like below, but using MQTT:

$ python3 pylips.py --command post --path menuitems/settings/current --body '{"nodes":[{"nodeid":2131230769}]}'
Sending POST request to https://192.168.200.33:1926/6/menuitems/settings/current
Request sent!
{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":1}}}],"version":0}

How can something like that be archived using MQTT? I tried posting the following to the topic_pylips:

$ mosquitto_pub -h 127.0.0.1 -u pylips -P XXXXXXXXXXXX -t pylips/Livingroom_TV/command -m '{"command":"post", "path":"menuitems/settings/current", "body":{"nodes":[{"nodeid":2131230769}]}}'

and I see the result in pylips' output:

Request sent!
{"styleName":"FOLLOW_VIDEO","isExpert":false,"menuSetting":"COMFORT","stringValue":"Comfort"}
Sending POST request to https://192.168.200.33:1926/6/menuitems/settings/current
Request sent!
{"values":[{"value":{"Nodeid":2131230774,"Controllable":true,"Available":true,"string_id":"Switch on","data":{"value":false}}}],"version":0}
{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":1}}}],"version":0}

but how can I receive the API reply THROUGH MQTT?

[eslavnov]

What is different? I don't see any difference between them... But nevermind, now pylips is working 👍

Sorry, I've misread your commands, they are indeed the same!

I saw somewhere on pylips' output a Please provide a 'path' argument error.

Yep, I am an idiot :) Thanks for noticing! I really need to research how to write tests in Python...

Trying mosquitto_pub -h 127.0.0.1 -u pylips -P XXXXXXXXXXXX -t pylips/Livingroom_TV/command -m '{"command":"ambilight_brightness", "body":{"value":10}}' doesn't work, but my python knowledge is limited...

Hmm, that's strange - it works for me, I've just tested it. Is the message actually getting published? (use mosquitto_sub to check your topic).

but how can I receive the API reply THROUGH MQTT?

Ah, I see what you mean, but I am not sure how this can be implemented. Since you are just publishing a message over MQTT, you can't really get anything back. I've envisioned this purely as a control method (e.g., "set the brightness to X") as opposed to "get the current brightness" (with the exception of some status items that I manually collect like power, mute, etc.). Theoretically speaking, I can publish a response as an MQTT message on a separate topic, for example, but how are you going to map a specific response to a specific command? If I dump all raw responses into the same topic, it will be a mess of different responses/formats. Creating a separate topic for each command is also not an option (we have 80+ commands). Finally, I could send responses in a format like {"your_command":{...}, "your_response":{...} - this could work, but you still need to write some logic on your side to parse it correctly, so it also kinda sucks.

I would recommend just using the manual mode with --verbose False for such use-cases:

python3 pylips.py --command post --path menuitems/settings/current --body '{"nodes":[{"nodeid":2131230769}]}' --verbose False

{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":10}}}],"version":0}

But if you have any better ideas how to handle replies over MQTT, I am all ears...

[mtzro2003]

But if you have any better ideas how to handle replies over MQTT, I am all ears...

One idea would be to design some configurable (by user) status message... Something like you have in available_commands.json where someone can add at any time his own commands. Could be interesting... As I'm out of town for the next week, I'll have time to think about it and maybe come up with a proposal. :smile: If not feasible, then the pylips in command line mode is the only way - which is not at all bad.

Once again, excellent job! Many-many-many thanks!!! :laughing:

On Mon, Apr 29, 2019 at 10:50 AM Evgeny Slavnov notifications@github.com wrote:

What is different? I don't see any difference between them... But nevermind, now pylips is working 👍

Sorry, I've misread your commands, they are indeed the same!

I saw somewhere on pylips' output a Please provide a 'path' argument error.

Yep, I am an idiot :) Thanks for noticing! I really need to research how to write tests in Python...

Trying mosquitto_pub -h 127.0.0.1 -u pylips -P XXXXXXXXXXXX -t pylips/Livingroom_TV/command -m '{"command":"ambilight_brightness", "body":{"value":10}}' doesn't work, but my python knowledge is limited...

Hmm, that's strange - it works for me, I've just tested it. Is the message actually getting published? (use mosquitto_sub to check your topic).

but how can I receive the API reply THROUGH MQTT?

Ah, I see what you mean, but I am not sure how this can be implemented. Since you are just publishing a message over MQTT, you can't really get anything back. I've envisioned this purely as a control method (e.g., "set the brightness to X") as opposed to "get the current brightness" (with the exception of some status items that I manually collect like power, mute, etc.). Theoretically speaking, I can publish a response as an MQTT message on a separate topic, for example, but how are you going to map a specific response to a specific command? If I dump all raw responses into the same topic, it will be a mess of different responses/formats. Creating a separate topic for each command is also not an option (we have 80+ commands). Finally, I could send responses in a format like {"your_command":{...}, "your_response":{...} - this could work, but you still need to write some logic on your side to parse it correctly, so it also kinda sucks.

I would recommend just using the manual mode with --verbose False for such use-cases:

python3 pylips.py --command post --path menuitems/settings/current --body '{"nodes":[{"nodeid":2131230769}]}' --verbose False

{"values":[{"value":{"Nodeid":2131230769,"Controllable":true,"Available":true,"string_id":"Brightness","data":{"value":10}}}],"version":0}

But if you have any better ideas how to handle replies over MQTT, I am all ears...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/eslavnov/pylips/issues/9#issuecomment-487481637, or mute the thread https://github.com/notifications/unsubscribe-auth/AAQWPIVB2JKXBGZWS524YT3PS2SC7ANCNFSM4HEHULBA .

[eslavnov]

I'll close this issue for now, please create a separate one with your proposal when you come up with one. Thanks!