enwi / hueplusplus

A simple C++ library to control Philips Hue lights on Linux, MacOS, Windows, Espressif ESP32 SDK and Arduino. Full documentation at
https://enwi.github.io/hueplusplus/
GNU Lesser General Public License v3.0
55 stars 22 forks source link

Support Hue Entertainment API #60

Closed CalcProgrammer1 closed 3 years ago

CalcProgrammer1 commented 4 years ago

I'm going to have a shot at implementing the Entertainment API. I will have a V2.1 bridge by the end of this week. The V2 bridge is required for the Entertainment API (meaning it is unsupported on the V1 bridge I already have). The API is documented here (requires login):

https://developers.meethue.com/develop/hue-entertainment/philips-hue-entertainment-api/

It looks to be a UDP protocol where one UDP packet contains up to 10 lights worth of data and can be sent up to 25 times per second. You have to request a client key when pairing with the bridge (not just a username):

POST http://\<ip-address>/api:

{
    "devicetype":"myapplication#myiphone",
    "generateclientkey":true
}

Response:

[
    {
         "success":{
              "username":"myzFXhsLU5Wg10eBithGE-LFikgjC7Q7SEGZsoEf",
              "clientkey":"E3B550C65F78022EFD9E52E28378583"
                   }
     }
]

The Entertainment API uses DTLS encrypted UDP messages. The message format looks like this:

image

It looks like this API allows true RGB control of the lights.

Before sending UDP messages you must enable the streaming mode on the group with a JSON message:

Set the enable streaming flag to true, using PUT to

HTTP://<ip-address>/api/<username>/groups/<identifier of entertainment group>

With body:

{"stream":{"active":true}}

Only one stream can be active for an entertainment area, the owner field in the entertainment area indicates the application which activated the stream and is streaming towards the area. I will edit this issue description with more information as I research it. Priorities are to figure out what DTLS encryption entails and find a simple library that can perform it, then dig into the bridge setup code to make sure we can get that client key. After that I will write a function for sending Entertainment Mode packets and hook it into my OpenRGB application to get a demonstration working.

OpenRGB is my open source, cross-platform application for synchronizing gaming peripherals and hardware and it would be awesome to synchronize Hue lights with other RGB hardware and integrate with game effects, music visualization, and more.

Tasks:

All tasks required to get Entertainment Mode working complete and pull request opened, but code is in need of a lot of cleanup still.

enwi commented 4 years ago

@CalcProgrammer1 Thanks for the great research, we were also planning on implementing the streaming API, but actually haven't added an issue for it yet. Also thanks again for tackling the task of implementing it yourself, that would be a huge help 😄

CalcProgrammer1 commented 4 years ago

Got my V2 bridge today and set up an entertainment zone using the app. Hopefully I can start working on this tonight or this weekend.

Edit: Having some weird issues connecting. I've tried both master and development builds of hueplusplus (in OpenRGB) and the same code that connected fine to my V1 bridge is throwing an exception when I try to connect to the V2 bridge (by entering its IP and MAC). My Home Assistant instance connected fine to both though.

Edit: Got the bridge working, but had to comment out these lines in WinHttpHandler.cpp. I'm not sure if this is just a new issue that showed up unrelated to the bridge or what.

    // shutdown the connection for sending since no more data will be sent
    // the client can still use the ConnectSocket for receiving data
    //if (shutdown(connect_socket, SD_SEND) == SOCKET_ERROR)
    //{
    //    int err = WSAGetLastError();
    //    std::cerr << "WinHttpHandler: shutdown failed: " << err << std::endl;
    //    throw(std::system_error(err, std::system_category(), "WinHttpHandler: shutdown failed"));
    //}

Now on to the important stuff - getting Entertainment mode to work.

Edit: We will need a DTLS library. mbedtls looks like it is light enough to include in a project and build from source maybe?

Here is a DTLS example: https://github.com/ARMmbed/mbedtls/blob/development/programs/ssl/dtls_client.c

Edit: Got a client key! Need to add API version checks, but getting the client key was easy enough by modifying the username request.

Started a branch for entertainment mode https://github.com/CalcProgrammer1/hueplusplus/tree/feature_entertainment_mode

This code works for finding Entertainment groups:

        /*-------------------------------------------------*\
        | Get all groups from the bridge                    |
        \*-------------------------------------------------*/
        std::vector<std::reference_wrapper<hueplusplus::Group>> groups = bridge.groups().getAll();

        if(groups.size() > 0)
        {
            /*-------------------------------------------------*\
            | Loop through all available groups and check to    |
            | see if any are Entertainment groups               |
            \*-------------------------------------------------*/
            for(unsigned int group_idx = 0; group_idx < groups.size(); group_idx++)
            {
                if(groups[group_idx].get().getType() == "Entertainment")
                {

                }
            }
        }

Edit: Added a function to request streaming. Returns true if the bridge confirms that streaming is active for the given group.

Been messing around with mbedtls...I think I have it connecting to the UDP server and handshaking, going to try sending a test packet.

Edit again: Got it! I was able to get the lights to respond to the entertainment packet!

            unsigned    char buf[] = { 'H',  'u',  'e',  'S',  't',  'r',  'e',  'a',  'm',     //HueStream
                                       0x01, 0x00,                                              //Version
                                       0x00,                                                    //Sequence ID
                                       0x00, 0x00,                                              //Reserved
                                       0x00,                                                    //Color space (RGB)
                                       0x00,                                                    //Reserved

                                       0x00,                                                    //Type (Light)
                                       0x00, 0x03,                                              //ID of light (1)
                                       0xFF, 0xFF,                                              //Red (16-bit)
                                       0x00, 0x00,                                              //Green (16-bit)
                                       0x00, 0x00,                                              //Blue (16-bit)
                                     };

                        ret = mbedtls_ssl_write(&ssl, (const unsigned char *)buf, sizeof(buf));
CalcProgrammer1 commented 4 years ago

I have smooth animation working now in OpenRGB! I wrote the code in my OpenRGB project for now, so I need to move the logic over to hueplusplus with some sort of Entertainment mode object.

The logic is here:

https://gitlab.com/CalcProgrammer1/OpenRGB/-/blob/542b338b069d0b06b2bc48d5774b9762417b6db7/Controllers/PhilipsHueController/PhilipsHueEntertainmentController.cpp

It creates and fills in a buffer based on the number of lights in the Entertainment group and signals the bridge to start and stop. I added an automatic refresher thread in OpenRGB as the bridge drops out of Entertainment mode after 10 seconds of inactivity. I'm not sure if this is something we want as part of hueplusplus or to leave up to implementing projects. My implementation updates a timer every time the program calls the LED update function (which sends a UDP message to the bridge). A background thread checks if this timer exceeds 5 seconds and if it does, it triggers the LED update function automatically to keep the bridge in Entertainment mode.

With this working, I went ahead and ordered all new Philips Hue color downlights for my room.

enwi commented 4 years ago

Edit: Got the bridge working, but had to comment out these lines in WinHttpHandler.cpp. I'm not sure if this is just a new issue that showed up unrelated to the bridge or what.

We saw that some users had this issue before, but could not replicate it ourselves, so I will test this again and make sure if I can replicate it or not.

I'm not sure if this is something we want as part of hueplusplus or to leave up to implementing projects.

I would suggest that this should be part of the project using hueplusplus, so the user can decide how he will implement it.

Other than that, great work so far 😃

CalcProgrammer1 commented 4 years ago

I pushed the initial files to my fork of hueplusplus and updated OpenRGB to call into these new files. It's working, but the files are pretty dirty right now and need some cleanup. If you could check it out and give your inputs on the API that would be great.

https://github.com/CalcProgrammer1/hueplusplus/commit/1154ef328e50004e3100e740a8b3ce1a86e74a66

Usage in OpenRGB:

https://gitlab.com/CalcProgrammer1/OpenRGB/-/commit/cde1940ef46fc710ffc6b9f168d20bbee6006375

Edit: Also, how do you want to go about including mbedtls?

enwi commented 4 years ago

@CalcProgrammer1 I added some comments in your commit. Also please open a WIP PR and we can discuss the code there

CalcProgrammer1 commented 4 years ago

Sounds good, I will address your comments and open a PR.

enwi commented 3 years ago

@CalcProgrammer1 We merged the changes into the development branch. So now you can officially use it for OpenRGB