search

project-chip / connectedhomeip

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.
https://buildwithmatter.com
Apache License 2.0
7.39k stars 1.98k forks source link

Deprecate chip-tool CLI and prioritize the python chip device controller as primary developer and certification tool #4334

Closed yunhanw-google closed 3 years ago

yunhanw-google commented 3 years ago

Problem

Currently we have two developer tool, chip-tool and python chip device controller, for device controller. For chip device controller, we see the below benefits -- Interactive Python interface to CHIPDeviceController.cpp which is used by iOS and android app -- Cross platform: OSX & Linux -- Controller & Commissioner functionality beyond what the chip-tool has -- whl package support for python wheel

To support two tools, it is a little bit painful for developer and test team.

In today's 1/12/2020 developer standup meeting, we agree to have this device controller as primary developer and certification tool beyond and would use this ticket to collect the missing stuff that chip device controller don't have but chip-tool has and further feature requests developer and test team wanna to have.

Proposed Solution

Deprecate chip-tool CLI, and missing feature support in chip device controller if necessary, and publish the existing CLI doc to github.

yunhanw-google commented 3 years ago

@andreilitvin @franck-apple @robszewczyk @rwalker-apple @bzbarsky-apple @mspang

kkasperczyk-no commented 3 years ago

@Damian-Nordic @LuDuda FYI

vivien-apple commented 3 years ago

I'm happy if at the end of the day we have a single developer tool!

On top of my head and to the best of my knowledge here are some stuffs that chip-tool contains that seems missing in the python version:

"-- Controller & Commissioner functionality beyond what the chip-tool has" -> Can you elaborate ? It may help me to catch some differences that I have forgotten.

yunhanw-google commented 3 years ago

I'm happy if at the end of the day we have a single developer tool!

Thank you!

On top of my head and to the best of my knowledge here are some stuffs that chip-tool contains that seems missing in the python version:

  • chip-tool CLI uses the BLE implementation from src/platform/. I would like to see that in the python CLI as well instead of https://github.com/project-chip/connectedhomeip/blob/master/src/controller/python/chip/ChipCoreBluetoothMgr.py since the platform BLE stack is what is running on the device.

In real device, we would not run CoreBluetooh and Bluez Central C++ implementation from platfrom BLE stack, since we would not support device vs device communication upon BLE in CHIP 1.0 scope, in device bluez side, we only run Bluez peripheral C++ implementation for device oobe/pairing. So in central test side, it is fine to put ChipCoreBluetoothMgr.py and ChipBluezMgr.py in device controller python CLI.

  • chip-tool relies on ChipDeviceand the new controller API, while the python device controller relies on the deprecated API.

Yep, Currently android test app is using Chip Device controller, and python test program is reusing what test app did. Agree, we can migrate the new ChipDevice API @erjiaqing

  • chip-tool has been written to do various checks against the content of the PacketBuffer that is received as a response to a command. It has let me find various bugs into the generated code. I don't know what is provided by the python device controller for such thing ? I understand that the IM may changes the content of those packets though, but until it lands, this is very convenient. if involving IM PacketBuffer, for different message, we provide CheckSchemaValidatity(https://github.com/project-chip/connectedhomeip/blob/master/src/app/MessageDef.cpp#L3408) to check packet buffer contents. @erjiaqing @vivien-apple we may check together on what useful check we can migrate to new tool.

  • chip-tool returns a failure if the response if unexpected. For example, if the cluster does not exists on the endpoint, or if the command is malformed. I'm not sure what the python device controller offers for that ?

for example, if command is malformed, IM's decoder(https://github.com/project-chip/connectedhomeip/blob/master/src/app/MessageDef.cpp#L3408) would check on it, IM is responsibie for message's format's check, and which is not device controller's responsiblity.

"-- Controller & Commissioner functionality beyond what the chip-tool has" -> Can you elaborate ? It may help me to catch some differences that I have forgotten. This tool would support BLE and IP rendezvous over PASE/CASE encryption, and WiFi/Thread/Fabric Provisioning, further it provides the python interface to IM so that it can easily send invoke Command, read/write cluster attribute against device over local BLE/IP connection.

vivien-apple commented 3 years ago

On top of my head and to the best of my knowledge here are some stuffs that chip-tool contains that seems missing in the python version:

  • chip-tool CLI uses the BLE implementation from src/platform/. I would like to see that in the python CLI as well instead of https://github.com/project-chip/connectedhomeip/blob/master/src/controller/python/chip/ChipCoreBluetoothMgr.py since the platform BLE stack is what is running on the device.

In real device, we would not run CoreBluetooh and Bluez Central C++ implementation from platfrom BLE stack, since we would not support device vs device communication upon BLE in CHIP 1.0 scope, in device bluez side, we only run Bluez peripheral C++ implementation for device oobe/pairing. So in central test side, it is fine to put ChipCoreBluetoothMgr.py and ChipBluezMgr.py in device controller python CLI.

Sorry by device I mean an iOS device. chip-tool simulates a controller, like the iOS device controller. And on iOS, src/platform/darwin is the used BLE stack.

"-- Controller & Commissioner functionality beyond what the chip-tool has" -> Can you elaborate ? It may help me to catch some differences that I have forgotten. This tool would support BLE and IP rendezvous over PASE/CASE encryption, and WiFi/Thread/Fabric Provisioning,

Ok. So for this part there is not difference to what chip-tool could be since it is basically replicating a controller.

further it provides the python interface to IM so that it can easily send invoke Command, read/write cluster attribute against device over local BLE/IP connection.

So the main difference is the Python interface then :) I guess this is an important difference for certification folks though !

vivien-apple commented 3 years ago

  • chip-tool has been written to do various checks against the content of the PacketBuffer that is received as a response to a command. It has let me find various bugs into the generated code. I don't know what is provided by the python device controller for such thing ? I understand that the IM may changes the content of those packets though, but until it lands, this is very convenient. if involving IM PacketBuffer, for different message, we provide CheckSchemaValidatity(https://github.com/project-chip/connectedhomeip/blob/master/src/app/MessageDef.cpp#L3408) to check packet buffer contents. @erjiaqing @vivien-apple we may check together on what useful check we can migrate to new tool.

  • chip-tool returns a failure if the response if unexpected. For example, if the cluster does not exists on the endpoint, or if the command is malformed. I'm not sure what the python device controller offers for that ?

for example, if command is malformed, IM's decoder(https://github.com/project-chip/connectedhomeip/blob/master/src/app/MessageDef.cpp#L3408) would check on it, IM is responsibie for message's format's check, and which is not device controller's responsiblity.

Agreed that this is not the controller responsibility to check for the message format.

I think my point was that I'm currently using chip-tool to checks if our ZAP integration works correctly, and so inside chip-tool there is a bunch of checks to validate that the response is appropriate, which let me do a bunch of scripts to quickly checks against it.

But maybe this bunch of checks is only useful for hacking CHIP but useless for other use cases...

yunhanw-google commented 3 years ago

Sorry by device I mean an iOS device. chip-tool simulates a controller, like the iOS device controller. And on iOS, src/platform/darwin is the used BLE stack. -- Yep, agree, for python device controller, we should replace ChipCoreBluetoothMgr.py with the same implementation as iOS app has.

Just create thee ticket https://github.com/project-chip/connectedhomeip/issues/4399

woody-apple commented 3 years ago

I think this should probably be tracked as a project... as the overall task here is to have the zap-generated-python-code fit all the needs for: • a hand rolled python implementation of CHIP, just wrap lib chip on each platform • a nice python API that is easy to use for testing/certification • a simple cli-tool that developers can use • supports all the current chip-tool features (like packet verification, @vivien-apple - what else?)

jeonghwan7 commented 3 years ago

Hi, I think Linux Chip-Tool is very useful for Embedded Controller Device and Not all devices support Python in particular Embedded Control device. I hope that it can use Linux Chip-Tool CLI for Embedded Controller Device continuously.

yunhanw-google commented 3 years ago

Hi, @wissn84

Thanks for your comments. python chip device controller is particularly used in linux desktop/raspi/macbook and simulate what mobile phone's pairing behavior. May I know which embedded controller device do you use for chip-tool CLI, and for which scenario?

erjiaqing commented 3 years ago

Hi, I think Linux Chip-Tool is very useful for Embedded Controller Device and Not all devices support Python in particular Embedded Control device. I hope that it can use Linux Chip-Tool CLI for Embedded Controller Device continuously.

Python Device Controller as well as the chip-tool, are targeted for testing, both the chip-tool and the python script are providing bindings for human use and not for building real products for use. The python script can provide programming interface and can build complex logics.

For building CHIP controllers on embedded linux platforms, I would suggest build an app using APIs provided by the controller class (or the CHIP library) and link it into your project instead of calling chip-tool or the python controller.

mspang commented 3 years ago

Hi, I think Linux Chip-Tool is very useful for Embedded Controller Device and Not all devices support Python in particular Embedded Control device. I hope that it can use Linux Chip-Tool CLI for Embedded Controller Device continuously.

FWIW I raised this issue in one of the SWTT meeting as a downside of deprecating native chip-tool, but nobody else in attendance seemed concerned about it at that time.

jeonghwan7 commented 3 years ago

To use Python Controller, The Embedded Controller Device is needed Python 3 and Python Library. Both will be burden in view of embedded device. As you know the device capability is small. For the upper reason I think Linux Controller API and chip-tool should be considered.

Damian-Nordic commented 3 years ago

@wissn84 I think what we're discussing here is a generic testing tool used for development and automated testing and even if we go for Python CHIP Controller, Linux Controller API will still exist and anyone will be able to implement their minimal controller application for constrained devices with little effort. I wonder if there is a use case for generic testing tool for embedded devices. My thinking is that embedded devices would rather need a controller application tailored to a specific scenario (instead of an application supporting all possible clusters/commands), but let me know if I'm wrong and please describe your needs/use case.

woody-apple commented 3 years ago

I'm not 100% sure what this is covering at this point, initially this was to move the certification process over to python (which is going well) - but we have no plans to delete the C++ tool, it's a great way to test the C++ API, we'll soon have an Objective C tool, to test the Objective C API. Let's close this for now, given we have plenty of issues tracking the python tool remaining work - and it's close.