search

Seeed-Studio / wiki-documents

https://wiki.seeedstudio.com/Getting_Started Seeed Studio Wiki source code
https://wiki.seeedstudio.com/Getting_Started
GNU General Public License v3.0
107 stars 125 forks source link

[Page Add][Enhanced Function] SenseCAP Indicator applies the Matter 1.0 protocol #516

Closed MatthewJeffson closed 1 year ago

MatthewJeffson commented 1 year ago

This assignment has been successfully achieved 🎉

Thanks Tim and the wiki page is: https://wiki.seeedstudio.com/SenseCAP_Indicator_Application_Matter/

Overview

Timo614 commented 1 year ago

Hi there, I'd like to take on this task. I have a sensecap indicator and have my local environment setup for Matter devices also have Home Assistant Yellow setup with Matter Beta.

I'm imagining taking the indicator_basis example, duplicating it, and introducing the matter protocol component with a configuration screen for the QR code and then of course the sensors reporting. Let me know if you had anything else in mind or if you wanted me to approach this from some other angle. My email is timo614 @ gmail.com.

If there's value in the future I could do a guide on setting up a Xiao ESP32S3 to work over matter to control a https://www.seeedstudio.com/Grove-RGB-LED-Stick-10-WS2813-Mini.html as a lighting app adapted from their lighting app example. May make for a nice IOT related demo project for that board as well.

MatthewJeffson commented 1 year ago

I have a sensecap indicator and have my local environment setup for Matter devices also have Home Assistant Yellow setup with Matter Beta.

Hi! @Timo614 Wow! That is really nice! I will think through your proposal here and get back to you as soon as possible. đź‘Ś

MatthewJeffson commented 1 year ago

I'm imagining taking the indicator_basis example, duplicating it, and introducing the matter protocol component with a configuration screen for the QR code and then of course the sensors reporting.

Hi, @Timo614 Sorry to keep you waiting. Our current function that wants to be implemented are just like you said.

  1. We are trying to make the SenseCAP indicator connect to Matter Apps like Homekit or Google Home
  2. The data model may have multiple clusters which mainly achieves the sensors data uploading and multiple on-off control to the other sensors.
  3. Just an example, the control UI on the indicator can be something like:

Image

Hope that makes sense.

Regarding Xiao ESP32S3 controlling RGB LED, we can add it as well, as the enhanced function. Currently controlling multiple sensors is what we are looking for.

Regards, Matthew

Timo614 commented 1 year ago

@MatthewJeffson

I'm in the process of integrating (I started a bit before I sent the message as I wanted to first confirm I could get my environment setup and experiment with matter a bit). I am setting up the matter endpoints to have a temperature sensor and humidity sensor per the supported devices of https://developers.home.google.com/matter/supported-devices (I had hoped to add the co2 and voc as well but no given the limited support for those measurement types unfortunately had to limit to those two of the initial sensor set reporting).

I think I understand what you're saying but just want to confirm to make sure I'm on the same page. So those actuators being defined there for use with that UI are not actually ones on the RP2040 via the i2c port or anything like that but rather they are defined by me in the project logic such that they report their state in Google Home? Then they would be used further in automations by the user in Home Assistant, Google Home, etc. to control their home? So it's effectively a neat way to make a custom home controller?

So from a user perspective they would:

  1. Go to the tab and would see the QR code to provision
  2. Provision using their cell phone via Google Home, Home Assistant or whatever Matter hub device they are using
  3. Upon finishing provisioning the device will display something similar to the example above with on/off switches and a dimmable light controller to the UI
  4. The state of those buttons will be kept up to date with matter such that they can be used for home automation

Is that correct? Apologies if I misunderstood / wasn't sure if the intent is to pull from i2c for the additional sensors here being controlled or the automation route.

MatthewJeffson commented 1 year ago

Hello, Tim! @Timo614

Then they would be used further in automations by the user in Home Assistant, Google Home, etc. to control their home? So it's effectively a neat way to make a custom home controller?

Yeah, it is indeed how this project can be understood.

i. Go to the tab and would see the QR code to provision ii. Provision using their cell phone via Google Home, Home Assistant or whatever Matter hub device they are using iii. Upon finishing provisioning the device will display something similar to the example above with on/off switches and a dimmable light controller to the UI iv. The state of those buttons will be kept up to date with matter such that they can be used for home automation

The steps are perfect. That would be great if we achieve that.

If it is right for you, shall we get started. :D Best Regards, Matthew

Timo614 commented 1 year ago

Hey @MatthewJeffson that sounds great, do you happen to have the squareline project that generated the above UI? I can try to make something similar if an existing project doesn't exist just not as familiar with design tools as I am with programming so figured I'd ask in case one existed.

Everything sounds great to me though will move forward with this.

MatthewJeffson commented 1 year ago

Hi, Tim @Timo614 We are using SquareLine Studio and here is a wiki we wrote. Hope this can be helpful. Regards, Matthew

Timo614 commented 1 year ago

Hey Matthew,

I may need a bit of an extension here.

I have things mostly working but I have run into some issues with memory and may need to rework the way the indicator builds the UI such that it doesn't put it all in memory which may take me a bit to sort out.

I'm going to keep at it but I wanted to give you an update so that you're aware of the issue and that I'm still at it not just putting the ticket aside and idling.

Edit: I've resolved my memory issues and am continuing here. I should be done with the task in the next 24 hours or so and will update here once set.

Tim

MatthewJeffson commented 1 year ago

Wow! That sounds amazing! Thank you for keeping me updated. Can't wait to see your work:D Best regards, Matthew

Timo614 commented 1 year ago

PR for the demo itself: https://github.com/Seeed-Solution/SenseCAP_Indicator_ESP32/pull/24 Uploading a video now and will link once I have that in place showing it working. Will work on the wiki article and push up a PR as well next.

MatthewJeffson commented 1 year ago

I can see it now and couldn't be more amazed by that! :o We will have our AE to test it as well. Meanwhile, would you mind giving me your email so we can discuss the bonus:D That is really great work. Best Regards, Matthew

Timo614 commented 1 year ago

Hey Matthew,

My email is timo614 @ gmail.com, thank you!

I've pushed up some videos showing the demo: https://photos.google.com/share/AF1QipNSgP0x3VHC4D1SGsiRcTuWa9qzAcE7PdVfn3XSmnu2Wm38uMu8VvHdaM1O17W-fw?key=d1VPV3JkRlVyd3M4MzFmd1dpb1dvdEsxOGN1c3RR The videos are still processing so may return an error but if you click back later should resolve once Google processes them. I recorded two videos one showing the provisioning process and the other with the UI post provisioning. Let me know if this is off in any way from what was expected and can take a look further.

Will put together that wiki article now, thanks,

Tim

Timo614 commented 1 year ago

I've pushed up a wiki page here as well: https://github.com/Seeed-Studio/wiki-documents/pull/562 Let me know if you want me to expand on it in some way further.

Thanks! Tim

MatthewJeffson commented 1 year ago

@Timo614 Hello, Tim. Nicely done, I will check with the wiki article. Meanwhile, @Wvirgil123 here is one of our EE and he will be charged of testing your PR. Hi @Wvirgil123 . Here are the information regarding the Matter 1.0 on Indicator:

Timo614 commented 1 year ago

For the demo video I didn't try to make it professional / assumed someone else would make the official video for the wiki. Is the demo video that official video for the youtube page? If so I can remake one quickly today and is more cohesive or edit them together such that they are.

Apologies on the wiki. I will address your comments there.

Wvirgil123 commented 1 year ago

Hello, Timo @Timo614 , Next I will test your program.

When I compiled your project via (esp-idf: v5.0.1 esp-matter: v1.1 ), an error was reported. May I ask if there are other things that need to be paid attention to in the compilation environment? 

-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Users/virgil/.espressif/tools/xtensa-esp32s3-elf/esp-2022r1-11.2.0/xtensa-esp32s3-elf/bin/xtensa-esp32s3-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32s3
Solving dependencies requirements
CMake Error at /Users/virgil/seeed/esp-idf/tools/cmake/build.cmake:519 (message):
  ERROR: HTTP request error

Call Stack (most recent call first):
  /Users/virgil/seeed/esp-idf/tools/cmake/project.cmake:440 (idf_build_process)
  CMakeLists.txt:29 (project)
Timo614 commented 1 year ago

It's very strange that you're seeing an HTTP request error as part of the build process.

The only additional requirement introduced here was the esp-matter dependency.

One thing I noticed is when I build idf creates a managed_components directory and a dependencies.lock file. It appears to be the component manager's work: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-guides/tools/idf-component-manager.html

dependencies:
  espressif/esp_encrypted_img:
    component_hash: f4ecbedae038527ff252a96336cc848cb1eb231838cd9a9377acadff33e3c493
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 2.0.3
  espressif/esp_secure_cert_mgr:
    component_hash: 0a7b7f1c26fbd6fd8f49cc90f39203e3ab9cf334bbbf288bede9db745c35892c
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 2.3.1
  espressif/json_parser:
    component_hash: 2130029a8c005ba12476c5b5ae5c1c4589cb0c171f8100644d1ab883434eb91f
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 1.0.0
  espressif/mdns:
    component_hash: 53b22a3b01d0b61180369a5dab00e271f1e725164b6affbc73af85e80b043658
    source:
      service_url: https://api.components.espressif.com/
      type: service
    version: 1.2.0
  idf:
    component_hash: null
    source:
      type: idf
    version: 5.0.0
manifest_hash: 7d4d0f588cefbae0ea061d3be45ed79eb4273b451bd6cbbf0d67354da81553f9
target: esp32s3
version: 1.0.0

It appears to pull these components directly from espressif. I had assumed this was normal behavior as I didn't do anything explicitly to trigger it and part of the espressif build output. I'll look into if it's some side effect of how I'm running the c++ build.

Timo614 commented 1 year ago

Ah yeah these are requirements of esp-matter via the idf component manager, I think it may be the cause of the HTTP issue if you're having connectivity issues with their component API: https://github.com/espressif/esp-matter/blob/main/components/esp_matter/idf_component.yml

My best guess offhand though. I can try with another older mac I have to see if I can install from scratch at some point to see if I run into any issues as well but just tried a full clean again and removing all my associated build data and had no issue so I think it may related to these managed components from esp-matter's dependency.

I could alternatively remove the managed_components directory from the gitignore but thought to include it as it felt like build related output. Edit: idf.py fullclean removes this directory so I think it will need to be gitignored as otherwise it gets removed upon cleaning the build output.

Wvirgil123 commented 1 year ago

It may be the reason for the managed_components cache or IDF version switch, and now the compilation will not report an error。

Wvirgil123 commented 1 year ago

It seems like the commission failed and keeps prompting to wait, Do I need to commission on the same network?

Image

Wvirgil123 commented 1 year ago

Hi, I have a little doubt and would like to ask you

  1. Is it using bt for commissioning?
    1. I see that the code uses the "dimmable_plugin_unit" endpoint instead of "dimmer_switch" or "on_off_switch". If so, can I use the indicator UI switch to control other matter devices? Can Google Home or HomeKit bind the Indicator's Switch to the Matter light?
Timo614 commented 1 year ago

Hi there,

Yup, bluetooth is used for commissioning (not sure if the google home app will force it to be enabled on your phone's side although from the above photo it looks like you do have bt enabled).

I don't have homekit or google home unfortunately just the Home Assistant Yellow so can't really test there but Matter is meant to be compatible so long as the device types match.

Hmm the on_off_switch and on_off_plugin_unit lack level control so that one doesn't seem to correspond to the one listed on the google list: https://developers.home.google.com/matter/supported-devices

From this release version the following ones appear to have level control: https://github.com/espressif/esp-matter/blob/release/v1.1/components/esp_matter/esp_matter_endpoint.cpp

color_temperature_light, extended_color_light, dimmer_switch, color_dimmer_switch, dimmable_plugin_unit

I assumed the On/Off Plug-in Unit from the list of supported devices was the one associated with the dimmable plugin unit but it's not clear from the documentation so perhaps I was incorrect here. Apologies, since Home Assistant is fine assumed all would be.

I could try pushing up a version of the branch that uses dimmer_switch instead of dimmable_plugin_unit and we could see if it's compatible across both. I would have thought the plugin unit sounded closer to the devices on that device list as there's nothing specifically about a dimmer switch on that device list though so seems odd. Edit: I'm going to adjust it actually in the same branch and use the color_temperature_light as Google explicitly claims to support that device type.

I'll see if I have an older Google Home unit around I can use to test that ecosystem further.

Timo614 commented 1 year ago

@Wvirgil123 could you try pulling down the latest version, erasing the flash, and trying this build? Pushed up a change which I confirmed still works locally with my Home Assistant Yellow but hopefully will be more compatible here with Google Home if that was the issue.

It's using the color_temperature_light device in place of the dimmable_plugin_unit which is listed explicitly under their supported devices for Matter 1.0.

On my home assistant UI it shows some color related buttons for shifting the color but for the purposes of the demo that could be ignored as it still responds to the slider and on/off functionality via the interface.

I saw you previously posting in the esp-matter repository so assuming the issue is probably one on the device / demo side as you've previously commissioned using the API via this phone and home ecosystem right? I'll keep looking for another ecosystem device or see if I can borrow one from a friend to test this out locally, apologies again, wish I hadn't thrown all those Google Home devices out a few years ago (hoping I have one I missed somewhere laying around).

Timo614 commented 1 year ago

Hey @Wvirgil123, I still don't have access to a Google device but I was able to provision it locally (it just says I can't control it) using the latest code in the repo plus one extra additional step on the google side that's not needed for other controllers.

You have to visit: https://console.home.google.com/projects Once there create a Matter project and associate the attached values for the vid and pid:

matter-google-home-install

Photos of the cellphone flow for google home with that (shows some configuration options via Google's API it appears like the image used there): https://photos.app.goo.gl/eiLa9t2JRZv8CEBv8

From there I was able to provision the device to my network just couldn't connect it to control it as I don't have the google home device. Let me know if you run into any problems. I can't test the google side but I think it should be set now. One thing to note: the google side only allows you to select one of the devices as the one you're configuring (from the list there) although when I go to the test page I can see all three devices so it seems to have synced them all properly I just lack a control device to run through the tests on that page.

If the process works for you and you're able to login and control the device do you mind taking a photo of the screen? I'll add steps to the wiki to provision the device on Google's side as well for completeness sake. Sorry again for the trouble here, if it continues to be an issue it may be worth us putting this card back into the needs work pile and having someone with Google Home take on this task as I may be limited in what I can do from my end here.

Wvirgil123 commented 1 year ago

hi, thank you very much for your patience answer. The commissioning is successful( Previous failures were due to network instability.), but the prompt is not a matter-certified device. is this the solution to this problem, via create a Matter project and associate the attached values for the vid and pid.

Wvirgil123 commented 1 year ago

I did commissioning through GHSAFM app https://developers.home.google.com/samples/matter-app?hl=en I don't have a google device either, I will use homekit for testing next since I have a HomePod.

Wvirgil123 commented 1 year ago

I have tested many times through the GHSAFM app, and found that the probability of failure to add devices is quite high. the APP prompts that there is a problem, but the indicator device does not display the QR code anymore。 Problem log: https://pastebin.com/f9gMAFEU

Wvirgil123 commented 1 year ago

The commissioning is successful via homekit. But I found that the percentage of the slider does not correspond to that on the app, as shown in Figure 2. https://photos.app.goo.gl/6LKKK8vYXQQGJcwT8

Timo614 commented 1 year ago

What's strange is the device is showing it doesn't have a connection to the wifi and the clock is not updated as well so the commissioning process seems to be in a strange state. I'll debug into this today to see if there's something that stands out to me in the commissioning callback flows. Could you provide a log of the device around the time you're moving the slider? Wondering if it's calling the matter endpoints to update them or returning some error there of some form from the sdk.

In terms of the app itself it shows the QR code when it calls chip::DeviceLayer::Internal::ESP32Utils::IsStationProvisioned() so some step in the provisioning process is convincing the sdk at least that's provisioned if it's no longer showing the QR code from that point forward. I'll see about building that demo app from Google later today and trying it out, thank you for the link.

Timo614 commented 1 year ago

I've modified a few of the config variables I was adjusting to keep the memory in check for the commissioning process. Could you try again? I switched a few back to their defaults, hoping this may explain the connectivity issues you had with wifi, etc.

I was able to grab the apk a bit ago for that google home demo app and confirmed that I can:

I suspect that demo app by google doesn't support the ability to have multiple devices associated with the root node. I believe this is a normal use case though per: https://blog.espressif.com/matter-clusters-attributes-commands-82b8ec1640a0 where the demo shown is the light switch with multiple light switch nodes attached to the root node. I'm not sure -- I'll look around to see if this is an explicit limitation of the demo app or some sign of some misconfiguration. I wish I had a google ecosystem to confirm further. The apple ecosystem showing the devices makes me think it's not an issue there though.

For the interest of minimizing potential problem vectors I switched the door_lock to an on_off_switch as I noticed the door_lock had some random errors in the logs and there were those home assistant beta support issues for the type (it doesn't support it from the server side on their end yet).

Could you try deleting your sdkconfig, fullclean, and do a rebuild flash and monitor and see things work better now? If not I may be at a loss here but am willing to keep trying if you don't mind passing the log. Apologies again, this seemed like a fun task to take on as an open source project to do in my spare time outside of work so I wanted to grab it but regretting it now with all of these issues and not a great way to debug them.

Wvirgil123 commented 1 year ago

In the homekit test ecosystem (The program currently being tested has not been updated to the latest version. commit: d12c74ff1059e041404f36f196e0ba5891c2b130 ) 。

The wifi is connected, and the time can be synchronized by ntp, but the wifi shows that it is disconnected. device log: https://pastebin.com/rnus5Ar2 moving the slider log：

Image

Wvirgil123 commented 1 year ago

The button and slider on the current indicator are displayed as a color temperature light, but in fact the device is not a light, and it seems that it cannot control other matter devices. Currently we want to implement these buttons to control other matter devices, as shown in the figure below.

Image

after binding，indicator can control other lights.

Image

If this function can be realized, I think this demo is very meaningful. (Of course, google home currently does not support “On/Off Light Switch”, I don’t know if other ecosystems support it)

Timo614 commented 1 year ago

I think there may be limitations on what is possible with google home currently via https://developers.home.google.com/matter/get-started#unsupported_features Binding is listed as an unsupported feature for that environment. https://developers.home.google.com/matter/supported-devices#onoff_light_switches

Because the Google Home ecosystem does not expose the Matter Binding cluster to its controllers, end users cannot configure an On/Off Light Switch to control other Matter devices using the GHA. They can commission a Matter On/Off Light Switch, but they won't be able to do anything with it afterwards, other than delete it from the home.

An end user commissioning an On/Off Light Switch on the Google fabric naturally will want to configure it to control another device. At this time, there is no way for the user to do this in the Google Home ecosystem. The GHA can't bind an On/Off Light Switch to another device's endpoint(s), because the Google Home ecosystem does not expose the Matter Binding cluster.

Just so I understand it correctly what you're describing is more akin to what's shown here: https://github.com/espressif/esp-box/tree/master/examples/matter_switch where they use the chip tool to set the binding to other matter devices vs in this case where I'm just reporting the state so it can be automated against?

It looks doable from the example as it just involves:

But I don't think it would work in google home given the above warnings. Also apple has a similar issue: https://github.com/project-chip/connectedhomeip/issues/27652#issuecomment-1624250218

I may be mistaken but I think the only option is to either surface this as an end device like this and emulate it over automation (I did that in my demo) or by controlling the network and handling the chip tool access control related functionality as a controller to create that mapping (but then the demo would be less useful I think as it wouldn't be supported at all in the major ecosystems).

Timo614 commented 1 year ago

Apologies again here, I'll close out my PRs associated with this as it's not the direction desired and after some searching I'm not sure of a way to accomplish the desired state outside of controlling the binding which I can't find a way to do in those ecosystems (and if I wouldn't be able to test it). Perhaps this is something much easier to accomplish than my approach and someone with more experience with Matter will know of a way outside of what I was thinking above (that's limited by the ecosystems). Sorry again for the distraction here.

Wvirgil123 commented 1 year ago

@Timo614 Hi, Tim. I think you have done a great job and have implemented the function of matter. According to the information you provided, the Binding function does have limitations in the Google Home and Home Kit ecology. We can not implement this function in the early stage, and can indicate the current problems and limitations in the readme. We will realize it later when the ecology is further improved. Thank you so much for all you do.

Timo614 commented 1 year ago

Hey Virgil, I wanted to take one more attempt at improving it as well. The memory issues have been a concern since the start since the esp-matter sdk ends up using so much so I ended up having to try so many different approaches there. I've adjusted it one last time in the branch to:

I confirmed the behavior still works with matter and my home assistant.

I also switched the devices to be: extended_color_light and door_lock which appear to handle the dimmer and switch functionalities here and are on both the google home and apple lists from what I can tell so have a better chance of compatibility / maybe something that someone else can file a PR to address whatever remains there. Wondering if perhaps this will fix the issues you saw with the dimmer in case it was just an issue of compatibility device wise.

Sounds great.

Wvirgil123 commented 1 year ago

Hi Timo. Sorry, I was busy last week and did not continue to test the matter you submitted. I'm encountering a problem now. When using HA APP to add matter devices, errors keep appearing. But the device is already connected to wifi and displays the time interface.

device firmware commit: ccfec8953b016a46d94d8c80b0a6e3ad50c79c07 HA APP: 2023.4.6( Current latest)

https://photos.google.com/share/AF1QipOEojFVw_erXsPTeIVrI9GeYvoAi3Q9RPapn-Azd9z6k5oD6VtOmxgn1-IN25oJ0g/photo/AF1QipMfNSe0Koz-AeK4BAdGzs2vB7-LZZD38XlJVGkB

Timo614 commented 1 year ago

Hi Virgil,

I'll step through this again today with a clean install and see if I run into any issues (I won't have a chance until this evening my time but will as soon as I can). I had pushed up some modifications with ccfec8953b016a46d94d8c80b0a6e3ad50c79c07 that reduced the memory footprint of the device to avoid some of the provisioning errors that were cropping up before.

For my HA app I have 2023.8.2 as the version https://github.com/home-assistant/android/releases/tag/2023.8.2 Is this a HA app for Apple? Will try that version in case there is some sort of issue specific to the platform and the logic somehow.

Do you know what errors you were seeing? Were they just random provisioning related ones?

Timo614 commented 1 year ago

I was able to find the version you mentioned with the apple app: https://apps.apple.com/no/app/home-assistant/id1099568401

It looks like this version is from back in April. Wondering if perhaps I'm using some devices they later added support for that may be causing problems with this early iteration. I'll go through this evening (I have an ipad) and see what errors I run into to see what's possible to adjust here for the ipad case.

Timo614 commented 1 year ago

Hi Virgil, I ended up being able to get things together quickly this morning to test things out and was able to get things fully provisioned with the app: https://photos.app.goo.gl/Jn24rj6Wcw7GiDce6

Not sure offhand what's going on with your end and the provisioning process there as I was able to consistently do it from my attempts. Could you check what errors it sends when you monitor it?

Wvirgil123 commented 1 year ago

I am using the Android app. I must have made a mistake, the version of the APP is 2023.8.2-full( Current latest). HA is installed through docker, the version is 2023.4.6. Next, I will update HA and test it again.

Image

Image

Timo614 commented 1 year ago

Sounds good, I use HA Yellow but I doubt it'll make any difference there. I'll try setting up home assistant on a docker container using a raspberry pi 4 I have and going through the process from scratch there and seeing if there are any issues that arise. I ordered an Espressif Thread border router kit so can further test with that with that environment as well once it arrives.

Timo614 commented 1 year ago

Hi Virgil, I was able to get things working on a Raspberry Pi 4 4 GB running Ubuntu with matter and HA installed via a docker container. It did run into an error a few times as my phone connected to the docker install of HA itself just logging in from my phone to the docker install (not sure what was up with that but seemed flakier than the normal install) but after that had no issues provisioning just was just a bit slower than normal for me https://photos.app.goo.gl/B9ESqgCYzE88qzMDA