Add support for USB Knx Interfaces

[farmio]

It would be nice to support USB interfaces in addition to KNX/IP. Next to supporting users that don't own IP Interfaces Xknx could be used as USB - IP bridge in the future.

See KNX specification 9 - 3: Basic System and Components - Couplers §3 KNX USB Interface

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Please make sure to update to the latest version of xknx (or Home Assistant) and check if that solves the issue. Let us know if that works for you by adding a comment 👍 This issue has now been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.

[kistlin]

For the not yet so familiar ones like myself. The specification can be downloaded for free from KNX Specifications. A free account can be created and then the mentioned item has to be put in the basket. After the order is placed (free again) you can download the specification.

KNX uses the USB HID device class.

For those that want to just have a quick look, here are some online resources.

KNX protocol description

• How to implement KNX USB Interface device?

C++ implementation of the USB interface

• knx

USB HID specification

• Device Class Definition for Human Interface Devices (HID)
• HID Usage Tables FOR Universal Serial Bus (USB)

To talk to KNX USB devices there are different possibilities.

The python library pyusb uses libusb and there is activity trying to integrate it into asyncio in pyusb/pyusb#63. There is also the python library hidapi which is more HID specific.

@farmio, do you already have thoughts on how to integrate all this?

[farmio]

Hi 👋!

If one is more comfortable in looking at Java there is also the Calimero project having Knx USB support. https://github.com/calimero-project

@kistlin I personally do not have any thoughts on how to implement this, nor any plans to do it myself. If you like to do it, you are very welcome! Feel free to join the xknx Discord Server if you want to chat about implementation details.

[kistlin]

For now I have done some isolated work in the examples folder. It can be found in the following commit https://github.com/kistlin/xknx/commit/c6f70a662d0507b79f0f1c2c083bfa59dab63725 Branch feature/add_usb_interface_support examples/usb

As a USB interface I used the Siemens OCI702 USB - KNX Serviceinterface.

Sending bytes from a recorded valid KNX frame, seems to work. Also parsing of a USB HID frame into a more meaningful object seems straight forward. I feel comfortable in the lower parts of the communication.

@farmio do you know where in the specification I find information on the application layer? I got in my view a non-trivial KNX device, which has lots of KNX objects in the manual. I would be interested how these objects map to actual bytes on a lower level.

And maybe a description (reference to a document) of the workflow of typical first steps. For example assigning an address to a device. What bytes do we need to send? So that I don't have to reverse everything.

[farmio]

Hi!

For Application layer information I'd look at 03_03_07 Application Layer v01.06.02 AS - in xknx we have the xknx.telegram.apci module for the different services. This can be passed to a xknx.telegram.Telegram object as payload. See https://github.com/XKNX/xknx/blob/main/xknx/knxip/cemi_frame.py for how we use it to encode/decode a Telegram object to and from KNX/IP. Afaik USB doesn't necessarily use CEMI-Frames so you maybe would have to add the USB-pendant for this.

To map KNX group objects to bytes you need to know the used DPT and decode it accordingly. See the xknx.dpt module. We use xknx.device to abstract that. If what you are looking for is to reprogram the device, I'm afraid I can't help you. Afaik this is vendor/device specific.

Assigning an address to a device is currently not supported from xknx. There is currently a PR open for this.

From https://www.cs.hs-rm.de/~werntges/proj/knxusb/KSC2005-LinuxUSB.pdf I read there is an Application note KNXonUSB (AN0037) for KNX over USB. Maybe you'll have to requeset it from KNXA (my.knx.org support ticket).

[kistlin]

Ok, maybe back to focusing on integrating USB support.

What helped me was document 03_06_03 EMI_IMI v01.03.03 AS.pdf in combination with this image in 09_03 Basic and System Components - Couplers v01.03.03 AS.pdf.

There in the KNX USB Transfer Protocol Body is the cEMI/EMI1/EMI2 frame. And from your comment and the code, xknx implements atm. only the cEMI frame right?

If I look for example in 03_06_03 EMI_IMI v01.03.03 AS.pdf chapter 4.1.5.3.3 L_Data.req. This is the higher level interpretation of the KNX USB Transfer Protocol Body.

So the integration in a perfect world would then be, receive USB packets on a defined interface and concatenate the data packets together. Once a complete frame is received, check what frame type it is and pass it on to the already existing xknx implementation.

All the higher levels are transport independant, as it should be. Am I missing something?

[farmio]

Yes sounds reasonable. The L_Data.req example here is a CEMI Frame - so this is what you get by creating a xknx.telegram.Telegram and encapsule it in a cemi frame xknx.knxip.cemi_frame.CEMIFrame (with CEMIFrame.init_from_telegram or setting the telegram property).

    telegram = Telegram(
            destination_address=GroupAddress("1/0/15"),
            payload=GroupValueWrite(DPTBinary(0)),
        )
    )
    cemi = CEMIFrame.init_from_telegram(xknx, telegram, code=CEMIMessageCode.L_DATA_REQ, src_addr=xknx.own_address)

Then append cemi_frame.to_knx() to your payload - this should make your whole KNX USB Transfer Protocol Body. To parse received frames pass the KNX USB Transfer Protocol Body (cut the header) to CEMIFrame.from_knx()

cemi = CEMIFrame(xknx)
try:
    cemi.from_knx(raw[header_length:])
except UnsupportedCEMIMessage as err:
    logger.warning("CEMI not supported: %s", err)
    # handle error

(see eg https://github.com/XKNX/xknx/blob/main/xknx/knxip/routing_indication.py)

As far as I know Cemi support is mandatory, EMI1 adn EMI2 are optional, so going with Cemi schould be fine.

[kistlin]

What do you think of an integration where a KNXIPInterface or USBInterface object, depending on the ConnectionConfig passed in, is instantiated.

To test this integration I created a ConnectionConfigUSB. The changes in xknx/xknx.py are pretty minimal.

@@ -114,14 +116,19 @@ class XKNX:
     async def start(self) -> None:
         """Start XKNX module. Connect to KNX/IP devices and start state updater."""
         self.task_registry.start()
-        self.knxip_interface = KNXIPInterface(
-            self, connection_config=self.connection_config
-        )
-        logger.info(
-            "XKNX v%s starting %s connection to KNX bus.",
-            VERSION,
-            self.connection_config.connection_type.name.lower(),
-        )
+        if isinstance(self.connection_config, ConnectionConfig):
+            self.knxip_interface = KNXIPInterface(
+                self, connection_config=self.connection_config
+            )
+            logger.info(
+                "XKNX v%s starting %s connection to KNX bus.",
+                VERSION,
+                self.connection_config.connection_type.name.lower(),
+            )
+        if isinstance(self.connection_config, ConnectionConfigUSB):
+            self.knxip_interface = USBInterface(self, connection_config=self.connection_config)
+            logger.info("XKNX start logging on USB device (idVendor: 0x%04x, idProduct: 0x%04x)",
+                        self.connection_config.idVendor, self.connection_config.idProduct)
         await self.knxip_interface.start()
         await self.telegram_queue.start()
         if self.start_state_updater:

The switch example with USB would look like this. Only the instantiation of XKNX needs a change.

...
async def main():
    xknx = XKNX(connection_config=ConnectionConfigUSB(USBVendorId.SIEMENS_OCI702, USBProductId.SIEMENS_OCI702))
    await xknx.start()
    switch = Switch(xknx, name="TestOutlet", group_address="1/1/11")
...

All the existing code with default initialization would still work as expected.

[farmio]

Sure for an initial proof of concept implementation this sounds reasonable. Later we can always rename knxip_interface to knx_interface and maybe consolidate the ConnectionConfig classes somehow so the KNXInterface class handles connections to IP and USB in the io module.

To communicate with xknx devices it just needs to register as an xknx.io.Interface subclass at xknx.knxip_interface.interface(so send_telegram is called, and register the callback xknx.knxip_interface.telegram_received to put received GroupValue* telegrams into xknx.telegrams asyncio.Queue.

Is there a tunnelling-like ConnectRequest - ConnectResponse process in USB or is it connectionless?

[kistlin]

In 09_03 Basic and System Components - Couplers v01.03.03 AS.pdf it says

3.4.2 KNX tunnelling

For communication between a tool on a PC and a KNX device connected to the KNX network, KNX frames are tunnelled on the USB link using one of the EMI formats. The time-out for a KNX tunnelling is 1 s. This is, a KNX USB Interface Device shall be able to receive a tunnelling frame, transmit it on the KNX medium and send the local confirmation back to the tool within 1 s. This is a recommended value which shall be complied to under normal bus load.

3.5.3.2 Device feature services

Figure 21 – Device Feature Service ...

For a possible use, please refer to the features proposed in 3.5.3.3 “Device features” below. Please note that only the Device Feature Get service is confirmed by a Device Feature Response service. The Device Feature Set- and the Device Feature Info services shall not be answered by the receiver.

...

The time-out for the Device Feature Get service is 1 s. This is, a KNX USB Interface Device shall reply with a Device Feature Response frame within 1 s.

For me the relevant parts sound like request/response only. Or just sending something.

[farmio]

Yes it seems no TunnelingACK is sent on USB, which is good. I guess L_DATA.con confirmation CEMI frames will be received (and should be waited for) just like on IP Tunneling?

Is it possible to identify a USB device as KNX interface? see https://wiki.debian.org/HowToIdentifyADevice/USB do they share some common device protocol or class or description?

[kistlin]

As for the first question, the only thing I found atm. is in 03_06_03 EMI_IMI v01.03.03 AS.pdf

4.1.5.1 Flow Control

During treatment of a request that is not yet confirmed to the cEMI client, the cEMI Server shall accept a new request from the cEMI client. This is used e.g. for management requests (see clause 4.1.7) during an L_Data.req/L_Data.con cycle.

Which indicates that the server could receive multiple requests, but I guess it makes it easier to implement (maybe?) if we just wait on the response first.

To the second question in 09_03 Basic and System Components - Couplers v01.03.03 AS.pdf

3 KNX USB Interface

3.1 Introduction

3.1.1 Scope

Discovery and self-description mechanisms on USB level are not in the scope of this clause as well as mechanisms for configuring and establishing of a communication link between the USB host (PC) and the KNX USB Interface Device. These mechanisms are managed by the USB host protocol. Please refer to the corresponding USB specification documents.

And the USB HID specification defines that

4.1 The HID Class

The USB Core Specification defines the HID class code. The bInterfaceClass member of an Interface descriptor is always 3 for HID class devices.

So in my understanding there is no real possibility to scan the USB devices and be sure it is a KNX device.

A possibility is to enumerate all HID devices and check for a known set of vendor/product id's. These would probably grow over time.

[farmio]

See https://github.com/XKNX/xknx/issues/323 (and the link there if you speak German) for the L_Data.con flow control. This was just recently added to xknx. Worked before with the rate_limiter but now we can be (I hope) sure we never flood the receivers buffers even on high Bus load (or disabled rate_limit).

[kistlin]

Yes the reference in your ticket to 4.1.5.1 Flow Control in 03_06_03 EMI_IMI v01.03.03 AS.pdf seems to indicate, that we should wait for a con.

4.1.5.1 Flow Control

cEMI Client To keep the flow control for Data Link Layer services as simple as possible (this allows a simple flow control state machine in the cEMI client), it is recommended that: • a cEMI client sends a new Data Link Layer request only when the confirmation of the preceding request is received, or • a request-to-confirmation timeout is recognised; the recommended time-out for the cEMI client is 3 seconds. A cEMI client shall at any time be able to accept an indication message from the cEMI Server.

FYI your assumption in the forum about the meaning of con was right, by accident I read that part. 03_03_04 Transport Layer v01.02.02 AS.pdf

1.1 Communication Modes

Every communication mode shall provide specific Transport Layer Service Access Points (TSAPs) accessible via different Transport Layer services. Each of these services shall consists of three service primitives, the request(req), the confirm (con) and the indication (ind).

And just for me, we are speaking about this, so that a possible USB implementation doesn't make the same mistake? :) Also do you consider a refactoring so we don't have twice the logic and twice the tests for the same problem? Or not yet?

Because for now I focused on the sending of a telegram. I hoped to just put a received telegram into the queue an we are good :).

[farmio]

Also do you consider a refactoring so we don't have twice the logic and twice the tests for the same problem? Or not yet?

Not yet, since not every sent telegram needs this (eg. when using routing). The confirmation waiting function is currently not more than https://github.com/XKNX/xknx/blob/0.18.15/xknx/io/tunnel.py#L282-L296 (you wouldn't need the Tunnelling class since no ACK is required)

I hoped to just put a received telegram into the queue an we are good :).

I think this will work fine. 👍

[farmio]

Hey! Small update for

Also do you consider a refactoring so we don't have twice the logic and twice the tests for the same problem? Or not yet?

I did move some code in #841 that may ease the implementation of a USB-Tunnel (using the _Tunnel class. USB seems very similar to what is needed for TCP tunnels). Its not perfect as it still has some IP-specific attributes, but I guess its a start.

Marvin is working on a refactoring of the ConnectionConfig class - we'd like to use separate Dataclasses for the connection types with some minor config validation baked into it - let's see.

[kistlin]

Hello,

recently I wasn't that active. I'll try to take a look into the changes next week.

And if one of you is working with KNX over USB, I have commited a Wireshark KNX USB dissector.

This helped me big time in quickly understanding the message flow. Put the script in the B.4. Plugin folders and it should work. Maybe you have to select the protocol in the GUI.

[mikimikeCH]

Hi kistlin,

I' am trying to use your USB implementation of xknx. When I use your example_usb.py I get:

INFO:xknx.log:XKNX start logging on USB device (idVendor: 0x0908, idProduct: 0x02dc)
WARNING:xknx.usb:TODO: load libusb dll on Windows
INFO:xknx.usb:found 1 device(s)
INFO:xknx.usb:device 1
INFO:xknx.usb:    manufacturer  : Siemens (idVendor: 0x0908)
INFO:xknx.usb:    product       : OCI702 KNX Interface (idProduct: 0x02dc)
INFO:xknx.usb:    serial_number : 00FD10D01DB7
DEBUG:xknx.telegram:<Telegram direction="Outgoing" source_address="0.0.0" destination_address="1/1/11" payload="<GroupValueWrite value="<DPTBinary value="True" />" />" />
DEBUG:xknx.state_updater:StateUpdater stopping
DEBUG:xknx.telegram:<Telegram direction="Outgoing" source_address="0.0.0" destination_address="1/1/11" payload="<GroupValueWrite value="<DPTBinary value="False" />" />" />
DEBUG:xknx.log:Stopping TelegramQueue```

It seems to find the USB KNX Interface and passes the telegram. I have a BUS Monitor running in ETS 6 with an older Siemens OCI700 Interface, but there I don't receive any Telegram. Is the problem that I have a Warning "load libusb dll"?  

[kistlin]

Hello @mikimikeCH,

as long as it finds the device the warning shouldn't matter that much. I just didn't run it on Windows yet.

The bigger problem is that the implementation is not in a usable state. I never finished it. I don't have an actual setup that works, just the programmer as you have. And I ran out of holidays :).

Are you familiar with debugging? Else you could follow the code and look where sending fails. If it even reaches USBClient.send_telegram in xknx/io/usb_client.py.

[Golpe82]

Hi there, i changed a bit the code of @kistlin for using my usb iface of jung:

util.py

class USBVendorId(IntEnum):
    """ Vendor ID's """
    SIEMENS_OCI702 = 0x0908
    JUNG_2130USBREG = 0x135e

class USBProductId(IntEnum):
    """ Product ID's """
    SIEMENS_OCI702 = 0x02dc
    JUNG_2130USBREG = 0X0023 

......

example_switch.py

........
async def main():
    #xknx = XKNX(connection_config=ConnectionConfigUSB(usb_util.USBVendorId.SIEMENS_OCI702, usb_util.USBProductId.SIEMENS_OCI702))
    xknx = XKNX(connection_config=ConnectionConfigUSB(usb_util.USBVendorId.JUNG_2130USBREG, usb_util.USBProductId.JUNG_2130USBREG))
    await xknx.start()
    #switch = Switch(xknx, name="TestOutlet", group_address="1/1/11")
    switch = Light(xknx, name="TestOutlet", group_address_switch="1/2/30")
    await switch.set_on()
    await asyncio.sleep(10)
    await switch.set_off()
    await xknx.stop()

if __name__ == "__main__":
    asyncio.run(main())

and i have similar output like @mikimikeCH

DEBUG:asyncio:Using selector: EpollSelector
INFO:xknx.log:XKNX start logging on USB device (idVendor: 0x135e, idProduct: 0x0023)
INFO:xknx.usb:found 1 device(s)
INFO:xknx.usb:device 1
INFO:xknx.usb:    manufacturer  : ALBRECHT JUNG GMBH & CO. KG (idVendor: 0x135e)
INFO:xknx.usb:    product       : KNX-USB Data Interface (idProduct: 0x0023)
INFO:xknx.usb:    serial_number : None
DEBUG:xknx.telegram:<Telegram direction="Outgoing" source_address="0.0.0" destination_address="1/2/30" payload="<GroupValueWrite value="<DPTBinary value="True" />" />" />
DEBUG:xknx.state_updater:StateUpdater stopping
DEBUG:xknx.telegram:<Telegram direction="Outgoing" source_address="0.0.0" destination_address="1/2/30" payload="<GroupValueWrite value="<DPTBinary value="False" />" />" />
DEBUG:xknx.log:Stopping TelegramQueue

Let´s find time for debugging...

Also a rebase with main is needed from the branch of @kistlin , there are small conflicts

[kistlin]

@Golpe82 I pushed changes that integrate the existing implementation into the current main of XKNX. And some minor changes. When I now run the example my KNX interface at least blinks :). Does not mean it works yet.

[farmio]

Oh that is great! Still looking forward to having support for USB from xknx directly 😃

We did some changes allowing to directly return list[Telegram] from a received Telegram - to answer incoming request-response queries directly (for management).

Unfortunately we still have no perfectly clean separation of (OSI)Layers - so I guess there will still be some duplicated code in USB and IP branches...

If one of you has any direct questions regarding implementation details or such, don't hesitate to join our Discord server! 👍

[Golpe82]

Hi, I was playing a bit today. I made a project 2 years ago that translate http to KNX frames using a raspi and a kberry. For it I needed to build cemi frames but embedded in ft1.2 protocol. I suppose for usb the ft1.2 protocoll is also needed, not only the cemi frames: https://drive.google.com/file/d/1-B_LsezclvBC3x42pkxy9pLOOp0ll1YZ/view?usp=drivesdk

[Golpe82]

I mean I was commuticating with the kberry over tty serial port and usb is also serial. I suppose xknx access the bus over network layer but serial connection succeed over link layer I think and the frame needs the ft1.2 protocoll for it so far I understand

[Golpe82]

Take a look to apendixE (this time English): https://www.google.com/url?q=https://weinzierl.de/images/download/documents/baos/knx_baos_binary_protocol_v2_0.pdf&sa=U&ved=2ahUKEwjSiJHE6v34AhVagv0HHfhnBwAQFnoECAAQAg&usg=AOvVaw0yEF3lHYytLXW9xje22UoO

Or maybe am I confusing things? Ft1.2 is for uart. The question is if the usb interface transforms dataframe to uart itsself? Aka. Embeds the cemi in a ft1.2 alone?

[farmio]

Eg. Calimero lists FT1.2 and KNX USB as separate protocols. https://github.com/calimero-project/calimero-core

I couldn't find any mention of "FT1.2" in the Knx specifications pdfs 🧐

[Golpe82]

ok, so far i understand ft1.2 is only for TP/KNX BAOS Module 830 of Weinzierl (also kberry) https://www.knx.org/wAssets/docs/downloads/Marketing/Flyers/KNX-Development-Solutions/KNX-Development-Solutions_en.pdf

[mikimikeCH]

I had some success to write to a KNX Device Programmingmode On/Off and Restart all those commands worked.

DEBUG:xknx.log:sending: <Telegram direction="Outgoing" source_address="0.0.0" destination_address="0.2.220" payload="<PropertyValueWrite object_index="0" property_id="54" count="1" start_index="1" data="01" />" />
DEBUG:xknx.log:write 64 bytes: 01131800080010010300001100b06002b402dc0603d7003610010100000000000000000000000000000000000000000000000000000000000000000000000000
WARNING:xknx.log:Error: KNX bus did not respond in time (2 secs) to payload request for: 0.2.220
DEBUG:xknx.telegram:<Telegram direction="Outgoing" source_address="0.0.0" destination_address="0.2.220" payload="<PropertyValueWrite object_index="0" property_id="54" count="1" start_index="1" data="00" />" />
DEBUG:xknx.log:sending: <Telegram direction="Outgoing" source_address="0.0.0" destination_address="0.2.220" payload="<PropertyValueWrite object_index="0" property_id="54" count="1" start_index="1" data="00" />" />
DEBUG:xknx.log:write 64 bytes: 01131800080010010300001100b06002b402dc0603d7003610010000000000000000000000000000000000000000000000000000000000000000000000000000```

I think there is a problem in reading the response from the Bus, because i allways get this warning:

WARNING:xknx.log:Error: KNX bus did not respond in time (2 secs) to payload request for: 0.2.220

[kistlin]

Hello @mikimikeCH

yes that is because receiving is not implemented yet. But good to hear that sending works.

To be able to do reads and writes simultaneously, I have to look into how this can be done safely with pyusb. Async operations get discussed but are not implemented yet, mentioned earlier.

The python library pyusb uses libusb and there is activity trying to integrate it into asyncio in pyusb/pyusb#63. There is also the python library hidapi which is more HID specific.

There is also the issue pyusb/pyusb#301 which talks about sending/receiving at the same time.

Because async operations might take a while, I tend to use threads. But I first have to check, that it is safe to do so.

[farmio]

Do we explicitly need pyusb or would pyserial-asyncio or something like that do as well?

I guess wrapping the communication in a thread would also work - due to HA specific reasons we even already do that optionally with our IP interfaces.

[kistlin]

There are now threads that send and receive.

A non-obvious thing was in CEMIFrame when using from_knx on the object. It was default initialized with outgoing direction and then I ended up in a message loop :). This could be changed to a @classmethod and ask for the direction and other required information. Unless I understood that wrong :D.

Currently the USB implementation does not handle receiving fragmented messages. This will be next.

[farmio]

Yes right, CEMIFrame does not have a direction attribute - this is added to our Telegram abstraction when creating them. See https://github.com/XKNX/xknx/blob/9aa172c8f3e8df4fcf5f5f41f30559a7b456cbd9/xknx/io/routing.py#L76 Improvements of this system would be welcome 😉 (would need to be a standard method as it needs lot of self.)

For the fragmented frames have a look at the TCP transport - it has to handle these too, maybe you can use these methods: https://github.com/XKNX/xknx/blob/9aa172c8f3e8df4fcf5f5f41f30559a7b456cbd9/xknx/io/transport/tcp_transport.py#L77

[farmio]

Calimero maintains a list of Usb VendorID and ProductIDs. Maybe this can be useful to do some kind of auto-discovery? For a HA context this would have to go in the manifest.json afaik, but maybe you can/want to use it in xknx as I have seen there is an Enum containing 2 now. https://github.com/calimero-project/calimero-core/blob/master/resources/knxUsbVendorProductIds

[kistlin]

Currently I changed the USB device search. If one passes no arguments to ConnectionConfigUSB like in the updated version of examples/usb/example_light.py, then the ported list from Calimero is searched. If it finds multiple devices, the first one is used. If vendor and product id are passed, a specific search is performed. Further adaptions should be rather easy. Now the ConnectionConfigUSB during instantiation is merely used to tell which connection to use, TCP/IP or USB.

Another issue is, when I attached the KNX USB device for the first time I get a cEMI message code xknx can't handle. M_PropInfo.ind = F7h (03_06_03 EMI_IMI v01.03.03 AS.pdf) There is no code to handle this yet.

I'm not sure what you mean by HA context and manifest.json. You mean vendor and product id would go into the manifest.json? And these then would be read out of there and xknx configured accordingly?

[farmio]

I'm not sure what you mean by HA context and manifest.json. You mean vendor and product id would go into the manifest.json? And these then would be read out of there and xknx configured accordingly?

Yes exactly. Home Assistant has its own USB discovery methods based on matching vendor and product id in any manifest.json. This discovery happens even when the knx integration (and therefore xknx) is not even configured or loaded. As a result we could use these to create a ConnectionConfigUSB and start xknx. See https://developers.home-assistant.io/docs/creating_integration_manifest#usb

The M_PropInfo should be easy to add so it can be handled - if we even need to handle it. https://github.com/XKNX/xknx/blob/9aa172c8f3e8df4fcf5f5f41f30559a7b456cbd9/xknx/knxip/knxip_enum.py#L53

[farmio]

4.1.7.3.6 M_PropInfo.ind

The M_PropInfo.ind message shall be applied by the management server to send an event notification to the management client, e.g. about a changed management Property value.

After reception of the M_PropInfo.ind, the management client shall check whether the contained data is relevant to one or more of the management procedures it supports. If so, these procedures shall be called with the received data. If no, the message shall be ignored.

M_PropInfo.ind shall always be an unconfirmed service.

Seems save to not do much about that 🤣

[kistlin]

@farmio yeah, I added the message type and just ignore it.

In addition I configured a KNX device that has a push button and sends temperature measurements. I can receive the messages over USB and decode them. The temperature comes as two byte float and with DPTTemperature the conversion is fine.

So it is looking better and better. The next step is to add some long overdue unit tests. And maybe a better look at sending telegrams.

[farmio]

Hi @kistlin 👋!

I do have a USB Interface available for testing now and would like to help push this forward.

Unfortunately on my system I do have problems getting your current branch working. I did install libusb beforehand, but get

ERROR:xknx.usb:Could not detach kernel driver: [Errno 13] Access denied (insufficient permissions)

this may be a macOS specific problem though so I'll see if I have more luck in a Linux VM or something in the coming days. It seems to be able to find and get infos from the device at least.

Complete output of examples/usb/example_light.py with amended GA

```py DEBUG:asyncio:Using selector: KqueueSelector INFO:xknx.log:XKNX start logging on USB device (idVendor: 0x0000, idProduct: 0x0000) INFO:xknx.log:XKNX v2.2.0 starting tunneling connection to KNX bus. INFO:xknx.usb:found 1 device(s) INFO:xknx.usb:device 1 INFO:xknx.usb: manufacturer : Gira Giersiepen GmbH & Co. KG (idVendor: 0x135e) INFO:xknx.usb: product : KNX-USB Data Interface (idProduct: 0x0022) INFO:xknx.usb: serial_number : None INFO:root:Using device: DEVICE ID 135e:0022 on Bus 001 Address 001 ================= bLength : 0x12 (18 bytes) bDescriptorType : 0x1 Device bcdUSB : 0x110 USB 1.1 bDeviceClass : 0x0 Specified at interface bDeviceSubClass : 0x0 bDeviceProtocol : 0x0 bMaxPacketSize0 : 0x8 (8 bytes) idVendor : 0x135e idProduct : 0x0022 bcdDevice : 0x103 Device 1.03 iManufacturer : 0x1 Gira Giersiepen GmbH & Co. KG iProduct : 0x2 KNX-USB Data Interface iSerialNumber : 0x0 bNumConfigurations : 0x1 CONFIGURATION 1: 50 mA =================================== bLength : 0x9 (9 bytes) bDescriptorType : 0x2 Configuration wTotalLength : 0x29 (41 bytes) bNumInterfaces : 0x1 bConfigurationValue : 0x1 iConfiguration : 0x0 bmAttributes : 0x80 Bus Powered bMaxPower : 0x19 (50 mA) INTERFACE 0: Human Interface Device ==================== bLength : 0x9 (9 bytes) bDescriptorType : 0x4 Interface bInterfaceNumber : 0x0 bAlternateSetting : 0x0 bNumEndpoints : 0x2 bInterfaceClass : 0x3 Human Interface Device bInterfaceSubClass : 0x0 bInterfaceProtocol : 0x0 iInterface : 0x0 ENDPOINT 0x81: Interrupt IN ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x81 IN bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 ENDPOINT 0x2: Interrupt OUT ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x2 OUT bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 ERROR:xknx.usb:Could not detach kernel driver: [Errno 13] Access denied (insufficient permissions) DEBUG:xknx.telegram:" />" /> DEBUG:xknx.log:sending: " />" /> DEBUG:xknx.log:write 64 bytes: 0113130008000b010300001100bce000000400010081000000000000000000000000000000000000000000000000000000000000000000000000000000000000 WARNING:xknx.log:[Errno 13] Access denied (insufficient permissions) Exception in thread USBSendThread: Traceback (most recent call last): File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/threading.py", line 1016, in _bootstrap_inner self.run() File "/Users/meti/Documents/Dev/kistlin-xknx/xknx/usb/usb_send_thread.py", line 39, in run self.usb_device.write(hid_frame.to_knx()) File "/Users/meti/Documents/Dev/kistlin-xknx/xknx/usb/util.py", line 209, in write write_count = self._ep_out.write(data) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 408, in write return self.device.write(self, data, timeout) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 986, in write intf, ep = self._ctx.setup_request(self, endpoint) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 113, in wrapper return f(self, *args, **kwargs) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 229, in setup_request self.managed_claim_interface(device, intf) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 113, in wrapper return f(self, *args, **kwargs) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/core.py", line 178, in managed_claim_interface self.backend.claim_interface(self.handle, i) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/backend/libusb1.py", line 829, in claim_interface _check(self.lib.libusb_claim_interface(dev_handle.handle, intf)) File "/Users/meti/.pyenv/versions/3.10.8/lib/python3.10/site-packages/usb/backend/libusb1.py", line 604, in _check raise USBError(_strerror(ret), ret, _libusb_errno[ret]) usb.core.USBError: [Errno 13] Access denied (insufficient permissions) WARNING:xknx.log:[Errno 13] Access denied (insufficient permissions) DEBUG:xknx.state_updater:StateUpdater stopping DEBUG:xknx.telegram:" />" /> DEBUG:xknx.log:sending: " />" /> DEBUG:xknx.log:Stopping TelegramQueue DEBUG:xknx.log:stopping thread USBSendThread DEBUG:xknx.log:stopping thread USBReceiveThread DEBUG:xknx.log:USBSendThread stopped DEBUG:xknx.log:USBReceiveThread stopped ```

---

Ad https://github.com/XKNX/xknx/issues/581#issuecomment-1003399838

Also do you consider a refactoring so we don't have twice the logic and twice the tests for the same problem? Or not yet?

This is now done in #1137 - so you'd only have to pass and handle CEMIFrame instances, no Telegram anymore. See https://github.com/XKNX/xknx/blob/915344c0ff2c57b47ed314657b26a7e0cab4adc0/xknx/io/knxip_interface.py#L433-L441

[mikimikeCH]

Hi @farmio

If i remember right i had to "Install a device filter" when i wanted to access my USB-KNX Interface from Python.

Maybe on macOS you have to do it as well.

[kistlin]

Hey @farmio,

great to hear. A few weeks back I added a bunch of unit tests for assembling/splitting frames. I would also like to get an initial merge soon.

I switched regularly between Windows and Linux, but not on a Mac. On Linux you shouldn't forget udev rules if you want to use it as normal user and Windows should work once the right drivers are installed.

But that problem looks similar to pyusb/pyusb#208. They mention to use HIDAPI and the Cython binding. Maybe a migration is required to get it to work.

If you can, you could try on Windows or Linux first. To run some basic tests. And then either of us could look into making it work on a Mac. I have one and I see the same error.

[farmio]

So after searching the web for some hours I think it is a limitation / security feature of macOS that can't be worked around that easily. See https://github.com/libusb/libusb/issues/1014 Looking at libusb documenation it even recommends to use the hidapi rather than libusb directly. https://github.com/libusb/libusb/wiki/FAQ#user-content-Does_libusb_support_USB_HID_devices which also has up-to-date python bindings which look quite promising.

• https://pypi.org/project/hidapi/
• https://pypi.org/project/hid/

I think I'll take a stab on testing these.

[kistlin]

Ok. Then I'll leave it up to you for now :). It might also simplify the code around USB.

[farmio]

Allright! I have copied your branch to https://github.com/XKNX/xknx/tree/usb-interface-support @kistlin Feel free to open a (draft-)PR against main 🙂

[farmio]

So I did some testing on Windows 10 and had no luck - even with different drivers. libusb1 times out with usb:Operation not supported or unimplemented on this platform

libusb1

```py DEBUG:asyncio:Using proactor: IocpProactor INFO:xknx.log:XKNX start logging on USB device (idVendor: 0x0000, idProduct: 0x0000) INFO:xknx.log:XKNX v2.2.0 starting tunneling connection to KNX bus. INFO:xknx.usb:found 1 device(s) INFO:xknx.usb:device 1 INFO:xknx.usb: manufacturer : Gira Giersiepen GmbH & Co. KG (idVendor: 0x135e) INFO:xknx.usb: product : KNX-USB Data Interface (idProduct: 0x0022) INFO:xknx.usb: serial_number : None INFO:root:Using device: DEVICE ID 135e:0022 on Bus 002 Address 008 ================= bLength : 0x12 (18 bytes) bDescriptorType : 0x1 Device bcdUSB : 0x110 USB 1.1 bDeviceClass : 0x0 Specified at interface bDeviceSubClass : 0x0 bDeviceProtocol : 0x0 bMaxPacketSize0 : 0x8 (8 bytes) idVendor : 0x135e idProduct : 0x0022 bcdDevice : 0x103 Device 1.03 iManufacturer : 0x1 Gira Giersiepen GmbH & Co. KG iProduct : 0x2 KNX-USB Data Interface iSerialNumber : 0x0 bNumConfigurations : 0x1 CONFIGURATION 1: 50 mA =================================== bLength : 0x9 (9 bytes) bDescriptorType : 0x2 Configuration wTotalLength : 0x29 (41 bytes) bNumInterfaces : 0x1 bConfigurationValue : 0x1 iConfiguration : 0x0 bmAttributes : 0x80 Bus Powered bMaxPower : 0x19 (50 mA) INTERFACE 0: Human Interface Device ==================== bLength : 0x9 (9 bytes) bDescriptorType : 0x4 Interface bInterfaceNumber : 0x0 bAlternateSetting : 0x0 bNumEndpoints : 0x2 bInterfaceClass : 0x3 Human Interface Device bInterfaceSubClass : 0x0 bInterfaceProtocol : 0x0 iInterface : 0x0 ENDPOINT 0x81: Interrupt IN ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x81 IN bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 ENDPOINT 0x2: Interrupt OUT ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x2 OUT bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 DEBUG:xknx.usb:Operation not supported or unimplemented on this platform DEBUG:xknx.telegram: DEBUG:xknx.log:sending: DEBUG:xknx.log:write 64 bytes: 0113130008000b010300001100bce000002914010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Exception in thread USBSendThread: Traceback (most recent call last): File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner self.run() File "c:\Users\meti\dev\xknx\xknx\usb\usb_send_thread.py", line 39, in run self.usb_device.write(hid_frame.to_knx()) File "c:\Users\meti\dev\xknx\xknx\usb\util.py", line 209, in write write_count = self._ep_out.write(data) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 408, in write return self.device.write(self, data, timeout) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 989, in write return fn( File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\backend\libusb1.py", line 855, in intr_write return self.__write(self.lib.libusb_interrupt_transfer, File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\backend\libusb1.py", line 938, in __write _check(retval) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\backend\libusb1.py", line 602, in _check raise USBTimeoutError(_strerror(ret), ret, _libusb_errno[ret]) usb.core.USBTimeoutError: [Errno 10060] Operation timed out WARNING:xknx.log:Error: KNX bus did not respond in time (2.0 secs) to GroupValueRead request for: 5/1/20 Value: None - took 2.008 seconds DEBUG:xknx.state_updater:StateUpdater stopping DEBUG:xknx.log:Stopping TelegramQueue DEBUG:xknx.log:stopping thread USBSendThread DEBUG:xknx.log:stopping thread USBReceiveThread DEBUG:xknx.log:USBSendThread stopped DEBUG:xknx.log:USBReceiveThread stopped ```

whereas libusb0 ´[Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n'´

libusb0

```py DEBUG:asyncio:Using proactor: IocpProactor INFO:xknx.log:XKNX start logging on USB device (idVendor: 0x0000, idProduct: 0x0000) INFO:xknx.log:XKNX v2.3.0 starting tunneling connection to KNX bus. ERROR:xknx.usb:No USB backend found. Set XKNX_LIBUSB environment variable pointing to libusb-1.0.dll or install it to C:\Windows\System32 INFO:xknx.usb:found 1 device(s) INFO:xknx.usb:device 1 INFO:xknx.usb: manufacturer : Gira Giersiepen GmbH & Co. KG (idVendor: 0x135e) INFO:xknx.usb: product : KNX-USB Data Interface (idProduct: 0x0022) INFO:xknx.usb: serial_number : None INFO:root:Using device: DEVICE ID 135e:0022 on Bus 000 Address 001 ================= bLength : 0x12 (18 bytes) bDescriptorType : 0x1 Device bcdUSB : 0x110 USB 1.1 bDeviceClass : 0x0 Specified at interface bDeviceSubClass : 0x0 bDeviceProtocol : 0x0 bMaxPacketSize0 : 0x8 (8 bytes) idVendor : 0x135e idProduct : 0x0022 bcdDevice : 0x103 Device 1.03 iManufacturer : 0x1 Gira Giersiepen GmbH & Co. KG iProduct : 0x2 KNX-USB Data Interface iSerialNumber : 0x0 bNumConfigurations : 0x1 CONFIGURATION 1: 50 mA =================================== bLength : 0x9 (9 bytes) bDescriptorType : 0x2 Configuration wTotalLength : 0x29 (41 bytes) bNumInterfaces : 0x1 bConfigurationValue : 0x1 iConfiguration : 0x0 bmAttributes : 0x80 Bus Powered bMaxPower : 0x19 (50 mA) INTERFACE 0: Human Interface Device ==================== bLength : 0x9 (9 bytes) bDescriptorType : 0x4 Interface bInterfaceNumber : 0x0 bAlternateSetting : 0x0 bNumEndpoints : 0x2 bInterfaceClass : 0x3 Human Interface Device bInterfaceSubClass : 0x0 bInterfaceProtocol : 0x0 iInterface : 0x0 ENDPOINT 0x81: Interrupt IN ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x81 IN bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 ENDPOINT 0x2: Interrupt OUT ========================== bLength : 0x7 (7 bytes) bDescriptorType : 0x5 Endpoint bEndpointAddress : 0x2 OUT bmAttributes : 0x3 Interrupt wMaxPacketSize : 0x40 (64 bytes) bInterval : 0x2 DEBUG:xknx.usb:is_kernel_driver_active WARNING:xknx.log:[Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n' DEBUG:xknx.telegram: DEBUG:xknx.cemi:Outgoing CEMI: DEBUG:xknx.log:sending: DEBUG:xknx.log:write 64 bytes: 0113130008000b010300001100bce000002914010000000000000000000000000000000000000000000000000000000000000000000000000000000000000000 Exception in thread USBSendThread: Traceback (most recent call last): File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\threading.py", line 1009, in _bootstrap_inner self.run() File "c:\Users\meti\dev\xknx\xknx\usb\usb_send_thread.py", line 33, in run self.usb_device.write(hid_frame.to_knx()) File "c:\Users\meti\dev\xknx\xknx\usb\util.py", line 214, in write write_count = self._ep_out.write(data) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 408, in write return self.device.write(self, data, timeout) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 986, in write intf, ep = self._ctx.setup_request(self, endpoint) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 113, in wrapper return f(self, *args, **kwargs) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 229, in setup_request self.managed_claim_interface(device, intf) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 113, in wrapper return f(self, *args, **kwargs) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\core.py", line 178, in managed_claim_interface self.backend.claim_interface(self.handle, i) File "C:\Users\meti\AppData\Local\Programs\Python\Python310\lib\site-packages\usb\backend\libusb0.py", line 447, in _check raise USBError(errmsg, ret) usb.core.USBError: [Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n' WARNING:xknx.log:[Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n' WARNING:xknx.log:Error: KNX bus did not respond in time (2.0 secs) to GroupValueRead request for: 5/1/20 Value: None - took 2.002 seconds DEBUG:xknx.state_updater:StateUpdater stopping WARNING:xknx.log:[Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n' WARNING:xknx.log:L_DATA_CON Data Link Layer confirmation timed out for DEBUG:xknx.log:Stopping TelegramQueue DEBUG:xknx.log:stopping thread USBSendThread DEBUG:xknx.log:stopping thread USBReceiveThread DEBUG:xknx.log:USBSendThread stopped DEBUG:xknx.log:USBReceiveThread stopped ```

Can you give me a hint of how to get that running on Windows? What are the right drivers?

I'll refactor to use CEMIFrame instead of Telegram meanwhile - but I guess this will be more straight forward if I could test it.

Edit: under Ubuntu VM I don't get the UDEV rule working. But with sudo, at least I don't get any error, the interface gets found and according to logs it seems a telegram is sent. Nothing hits the bus though 🫤

USB is simple and reliable, they said 🤣

[kistlin]

So I did some testing on Windows 10 and had no luck - even with different drivers. libusb1 times out with usb:Operation not supported or unimplemented on this platform

libusb1 whereas libusb0 ´[Errno None] b'libusb0-dll:err [claim_interface] could not claim interface 0, invalid configuration 0\n'´

libusb0 Can you give me a hint of how to get that running on Windows? What are the right drivers?

Did you see _get_usb_backend in xknx\usb\util.py? It expects libusb1 to be at C:\Windows\System32\libusb-1.0.dll or you can set it with an environment variable XKNX_LIBUSB.

Yeah all these quirks. But HID might help.

[farmio]

Yes I saw that and changed the path to my .DLL that was provided by libusb_package and also a downloaded version from libusb.info. I'm a Windows noob - I probably forgot something obvious.

[kistlin]

If that doesn't help then it was me probably using Zadig and install WinUSB for my usb device. I just had to remove that, else hidapi would no longer find my device.

I would say let's forget about pyusb/libusb and just use hidapi. Too many inconveniences.

[kistlin]

@farmio I quickly hacked some hidapi support in. You might try it on your system. It's in my branch under usb-interface-support. Using the monitor example I don't get what I expect. But running the switch example, it seems to communicate.

[farmio]

I did also try Zadig 🤣 Ok, just tested your branch - hidapi doesn't raise any errors on macOS and Windows - so that's good. But it also doesn't actually send anything to the bus - so thats to improve 😬

[farmio]

Ok, so unfortunately it seems it's more work to get this running with my interface than I thought... according to this Device Feature Get - Response

hidapitester --vidpid 135e/0022 --open --send-output 1,0x13,0x09,0x00,0x08,0x00,0x01,0x0F,0x01,00,00,01 --read-input
Opening device, vid/pid: 0x135E/0x0022
Writing output report of 64-bytes...wrote 64 bytes:
 01 13 09 00 08 00 01 0F 01 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Reading 64-byte input report 0, 250 msec timeout...read 64 bytes:
 01 13 0B 00 08 00 03 0F 02 00 00 01 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Closing device

it looks to me that my interface doesn't support CEMI, but only EMI1 😭