search

libusb / hidapi

A Simple cross-platform library for communicating with HID devices
https://libusb.info/hidapi/
Other
1.68k stars 398 forks source link

HID reports of more than 64 bytes and libusb backend #274

Open JoergAtGithub opened 3 years ago

JoergAtGithub commented 3 years ago

At Mixxx, users reported problems with USB 2.0 High-Speed devices, which are limited to Full-Speed mode for some reasons. Under MacOS and Windows they operate as High-Speed device.

This is not only a performance limitation, but it has functional implications for the HID communication, because USB Full-Speed is limited to a package size of 64 Bytes instead of 1024 in High-Speed mode. The result is, that HIDAP returns a 79Byte input report correct on Windows and MacOS, but two reports on Linux (libusb backend), the first with correct report ID, the second with report ID 0x00.

How could a cross platform application using HIDAPI handle this:

Linux-Output (libusb): Debug [Controller]: Traktor Kontrol S4 MK3 A79A_3: t:15558 ms, 64 bytes: 02 6E 08 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Debug [Controller]: Traktor Kontrol S4 MK3 A79A_3: t:15559 ms, 15 bytes: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

This is the lsusb output: Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 3 bAlternateSetting 0 bNumEndpoints 2 bInterfaceClass 3 Human Interface Device bInterfaceSubClass 0 bInterfaceProtocol 0 iInterface 11 Traktor Kontrol S4 MK3 HID HID Device Descriptor: bLength 9 bDescriptorType 33 bcdHID 1.10 bCountryCode 0 Not supported bNumDescriptors 1 bDescriptorType 34 Report wDescriptorLength 673 Report Descriptors: UNAVAILABLE Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x83 EP 3 IN bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0040 1x 64 bytes bInterval 1 Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x02 EP 2 OUT bmAttributes 3 Transfer Type Interrupt Synch Type None Usage Type Data wMaxPacketSize 0x0040 1x 64 bytes bInterval 2 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 4 bAlternateSetting 0 bNumEndpoints 1 bInterfaceClass 255 Vendor Specific Class bInterfaceSubClass 189 bInterfaceProtocol 0 iInterface 12 Traktor Kontrol S4 MK3 BD Endpoint Descriptor: bLength 7 bDescriptorType 5 bEndpointAddress 0x03 EP 3 OUT bmAttributes 2 Transfer Type Bulk Synch Type None Usage Type Data wMaxPacketSize 0x0200 1x 512 bytes bInterval 1 Interface Descriptor: bLength 9 bDescriptorType 4 bInterfaceNumber 5 bAlternateSetting 0 bNumEndpoints 0 bInterfaceClass 254 Application Specific Interface bInterfaceSubClass 1 Device Firmware Update bInterfaceProtocol 1 iInterface 10 Traktor Kontrol S4 MK3 DFU Device Firmware Upgrade Interface Descriptor: bLength 9 bDescriptorType 33 bmAttributes 7 Will Not Detach Manifestation Tolerant Upload Supported Download Supported wDetachTimeout 250 milliseconds wTransferSize 64 bytes bcdDFUVersion 1.10 Device Qualifier (for other device speed): bLength 10 bDescriptorType 6 bcdUSB 2.00 bDeviceClass 0 bDeviceSubClass 0 bDeviceProtocol 0 bMaxPacketSize0 64 bNumConfigurations 1

Youw commented 3 years ago

Do I understand it correctly that the problem on Linux with libusb backend?

JoergAtGithub commented 3 years ago

Yes, it's reported by two independend Linux libusb users. I'm myself just relaying this, since I've neither the device nor Linux.

Youw commented 3 years ago

How could a cross platform application using HIDAPI handle this:

Can the speed or max. package size be determined using HIDAPI to report an error about the USB misconfiguration?

It is hard to define in a first place what is a "misconfiguration". From USB stack POV, if a device is successfully initialized in Full-Speed it is a successfull configuration. Otherwise it wouldn't be configured at all. How would a client application know what it should expect (unless it is a known device, expected to run in a specific mode)?

And one more time: hidapi is agnostic of the device bus most of the time. On Windows/macOS/hidraw communicationis done using an OS API and there is nothing USB-specific.

Can the application tell the device somehow to switch to USB High-Speed mode?

It is only negotiated by the OS/USB driver/stack and device itself. The application has no control of this. Each device always tries to initialize itself in a highest possible mode, unless explicitly required otherwise by the device FW/HW limitations.

Can HIDAPI check, if a report descriptor contains report sizes of more than 64Bytes for a device in Full-Speed mode?

Again no cross-platform solution is possible. And right now there is no API to get the descriptor. And the USB speed mode is ony easily can be checked by libusb backend.

The device's FW is way mo easily can check/know current USB configuration. If device's FW can be modified, I'd rather suggest to make an API and ask the device in which USB mode is it running.

JoergAtGithub commented 3 years ago

Modification of the the device firmware is not possible. The device is a commerical DJ controller, which is sold for use with the commercial DJ software Traktor Pro for Windows and MacOS. The users try to use it on the cross platform DJ software Mixxx under Linux, which is based on HIDAPI. Mixxx works with this device on MacOS and even on a Windows 10 VM under Linux, but not if under Linux with libusb backend.

Youw commented 3 years ago

What about hidraw backend, as anyone tried it with that device?

If there is no known issues with Windows/macOS, I think it would be feasible to perform an explicit check on Linux using libusb directly, even before trying to open it with hidapi.

JoergAtGithub commented 3 years ago

hidraw is not yet tried. Further investigation is needed to understand this specific case. But I think this is a general issue which could also occur with other HID devices, maybe also on other operating systems. Therfore it would be better to detect this somehow as error in hid_open instead of adding a check on application level.

Youw commented 3 years ago

That's what I'm trying to say: a USB/HID device running at Full-Speed - is not an error.

As for the inability to send packets larger than 64 bytes: this looks like an undesirable behavior of a specific device, but that's definitely not a HID issue. HID protocol is not design to send large packets in a first place. It is specifically designed to send many small packets (2/4/8/16 bytes of data), and that's what is being done by all devices I've seen. I've had one device that has been using 64 bytes data packets, and that was considered an edge case. For ceses when large data packets needs to be send between host and device, usually different protocols are used, like USB BULK.

Youw commented 3 years ago

Can HIDAPI check, if a report descriptor contains report sizes of more than 64Bytes for a device in Full-Speed mode?

After giving it a second though, we can call a case like this a "broken device" or "hardware missconfiguration" which can be detected by libusb backend of hidapi. If someone is going to develop a patch to detect such a thing for libusb backend, I don't see ~strong~ reasons not to accept it.

Be-ing commented 3 years ago

I have a Native Instruments Traktor Kontrol S4 Mk3, one of the affected devices. Here is the output of lsusb -t for the device:

    |__ Port 7: Dev 25, If 0, Class=Hub, Driver=hub/4p, 480M
        |__ Port 1: Dev 29, If 0, Class=Hub, Driver=hub/2p, 480M
            |__ Port 1: Dev 30, If 4, Class=Vendor Specific Class, Driver=, 480M
            |__ Port 1: Dev 30, If 2, Class=Audio, Driver=snd-usb-audio, 480M
            |__ Port 1: Dev 30, If 0, Class=Audio, Driver=snd-usb-audio, 480M
            |__ Port 1: Dev 30, If 5, Class=Application Specific Interface, Driver=, 480M
            |__ Port 1: Dev 30, If 3, Class=Human Interface Device, Driver=usbhid, 480M
            |__ Port 1: Dev 30, If 1, Class=Audio, Driver=snd-usb-audio, 480M

The HID component is at 480M speed, so I don't think the bug is in the kernel. The audio interface of the device works fine with Linux.

I don't know if the bug here is in libusb or hidapi's use of libusb. My hunch is the latter but I have no evidence to demonstrate that.

Youw commented 3 years ago

I don't believe libusb (as a result - hidapi too) has any control of how to initialize the device, select speed, etc. USB stack driver is responsible for that.

Do you have the same issue on Application level on your machine with this configuration?

@JoergAtGithub can you ask users who reported this issue to share the output of lsusb -t too?

Be-ing commented 3 years ago

It seems that libusb can only read a device's speed with libusb_get_device_speed() but has no API to set the speed.

Be-ing commented 3 years ago

Here is the lsusb -v output with formatting... the wMaxPacketSize 0x0040 1x 64 bytes for the HID endpoint is interesting...

Bus 001 Device 032: ID 17cc:1720 Native Instruments Traktor Kontrol S4 MK3
Couldn't open device, some information will be missing
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass          239 Miscellaneous Device
  bDeviceSubClass         2 
  bDeviceProtocol         1 Interface Association
  bMaxPacketSize0        64
  idVendor           0x17cc Native Instruments
  idProduct          0x1720 
  bcdDevice            0.60
  iManufacturer           1 Native Instruments
  iProduct                3 Traktor Kontrol S4 MK3
  iSerial                 2 69DDA7A8
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0124
    bNumInterfaces          6
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xc0
      Self Powered
    MaxPower                0mA
    Interface Association:
      bLength                 8
      bDescriptorType        11
      bFirstInterface         0
      bInterfaceCount         3
      bFunctionClass          1 Audio
      bFunctionSubClass       0 
      bFunctionProtocol      32 
      iFunction               0 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      1 Control Device
      bInterfaceProtocol     32 
      iInterface              3 
      AudioControl Interface Descriptor:
        bLength                 9
        bDescriptorType        36
        bDescriptorSubtype      1 (HEADER)
        bcdADC               2.00
        bCategory               8
        wTotalLength       0x0053
        bmControls           0x00
      AudioControl Interface Descriptor:
        bLength                 8
        bDescriptorType        36
        bDescriptorSubtype     10 (CLOCK_SOURCE)
        bClockID               41
        bmAttributes            3 Internal programmable clock 
        bmControls           0x07
          Clock Frequency Control (read/write)
          Clock Validity Control (read-only)
        bAssocTerminal          0
        iClockSource            9 
      AudioControl Interface Descriptor:
        bLength                 8
        bDescriptorType        36
        bDescriptorSubtype     11 (CLOCK_SELECTOR)
        bClockID               40
        bNrInPins               1
        baCSourceID(0)         41
        bmControls           0x03
          Clock Selector Control (read/write)
        iClockSelector          8 
      AudioControl Interface Descriptor:
        bLength                17
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             2
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bCSourceID             40
        bNrChannels             4
        bmChannelConfig    0x00000000
        iChannelNames          13 
        bmControls         0x0000
        iTerminal               6 
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID            20
        wTerminalType      0x0301 Speaker
        bAssocTerminal          0
        bSourceID               2
        bCSourceID             40
        bmControls         0x0000
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                17
        bDescriptorType        36
        bDescriptorSubtype      2 (INPUT_TERMINAL)
        bTerminalID             1
        wTerminalType      0x0201 Microphone
        bAssocTerminal          0
        bCSourceID             40
        bNrChannels             8
        bmChannelConfig    0x00000000
        iChannelNames          17 
        bmControls         0x0000
        iTerminal               0 
      AudioControl Interface Descriptor:
        bLength                12
        bDescriptorType        36
        bDescriptorSubtype      3 (OUTPUT_TERMINAL)
        bTerminalID            22
        wTerminalType      0x0101 USB Streaming
        bAssocTerminal          0
        bSourceID               1
        bCSourceID             40
        bmControls         0x0000
        iTerminal               7 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              4 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        1
      bAlternateSetting       1
      bNumEndpoints           2
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              4 
      AudioStreaming Interface Descriptor:
        bLength                16
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink           2
        bmControls           0x00
        bFormatType             1
        bmFormats          0x00000001
          PCM
        bNrChannels             4
        bmChannelConfig    0x00000000
        iChannelNames          13 
      AudioStreaming Interface Descriptor:
        bLength                 6
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bSubslotSize            4
        bBitResolution         24
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x01  EP 1 OUT
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x00d0  1x 208 bytes
        bInterval               1
        AudioStreaming Endpoint Descriptor:
          bLength                 8
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bmControls           0x00
          bLockDelayUnits         2 Decoded PCM samples
          wLockDelay         0x0008
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x82  EP 2 IN
        bmAttributes           17
          Transfer Type            Isochronous
          Synch Type               None
          Usage Type               Feedback
        wMaxPacketSize     0x0004  1x 4 bytes
        bInterval               4
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              5 
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        2
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         1 Audio
      bInterfaceSubClass      2 Streaming
      bInterfaceProtocol     32 
      iInterface              5 
      AudioStreaming Interface Descriptor:
        bLength                16
        bDescriptorType        36
        bDescriptorSubtype      1 (AS_GENERAL)
        bTerminalLink          22
        bmControls           0x00
        bFormatType             1
        bmFormats          0x00000001
          PCM
        bNrChannels             8
        bmChannelConfig    0x00000000
        iChannelNames          17 
      AudioStreaming Interface Descriptor:
        bLength                 6
        bDescriptorType        36
        bDescriptorSubtype      2 (FORMAT_TYPE)
        bFormatType             1 (FORMAT_TYPE_I)
        bSubslotSize            4
        bBitResolution         24
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            5
          Transfer Type            Isochronous
          Synch Type               Asynchronous
          Usage Type               Data
        wMaxPacketSize     0x01a0  1x 416 bytes
        bInterval               1
        AudioStreaming Endpoint Descriptor:
          bLength                 8
          bDescriptorType        37
          bDescriptorSubtype      1 (EP_GENERAL)
          bmAttributes         0x00
          bmControls           0x00
          bLockDelayUnits         2 Decoded PCM samples
          wLockDelay         0x0008
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        3
      bAlternateSetting       0
      bNumEndpoints           2
      bInterfaceClass         3 Human Interface Device
      bInterfaceSubClass      0 
      bInterfaceProtocol      0 
      iInterface             11 
        HID Device Descriptor:
          bLength                 9
          bDescriptorType        33
          bcdHID               1.10
          bCountryCode            0 Not supported
          bNumDescriptors         1
          bDescriptorType        34 Report
          wDescriptorLength     673
         Report Descriptors: 
           ** UNAVAILABLE **
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x83  EP 3 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               1
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x02  EP 2 OUT
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0040  1x 64 bytes
        bInterval               2
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        4
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass       255 Vendor Specific Class
      bInterfaceSubClass    189 
      bInterfaceProtocol      0 
      iInterface             12 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x03  EP 3 OUT
        bmAttributes            2
          Transfer Type            Bulk
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0200  1x 512 bytes
        bInterval               1
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        5
      bAlternateSetting       0
      bNumEndpoints           0
      bInterfaceClass       254 Application Specific Interface
      bInterfaceSubClass      1 Device Firmware Update
      bInterfaceProtocol      1 
      iInterface             10 
      Device Firmware Upgrade Interface Descriptor:
        bLength                             9
        bDescriptorType                    33
        bmAttributes                        7
          Will Not Detach
          Manifestation Tolerant
          Upload Supported
          Download Supported
        wDetachTimeout                    250 milliseconds
        wTransferSize                      64 bytes
        bcdDFUVersion                   1.10
Be-ing commented 3 years ago

How do I tell hidapi to use the hidraw backend instead of libusb? I don't see any documentation for this. ./configure --help has no information. It looks like make builds both hidraw and libusb. If I uninstall the libusbx-devel package on Fedora, hidapi fails to build.

Youw commented 3 years ago

You can still have both hidraw and libusb backends of hidapi installed and use either one of them. Technically those are two different libraries (even though with identical public API/ABI). If you want to switch the backend for you application, you need to link against libhidapi-hidraw instead of libhidapi-libusb.

If you have a prebuilt application, it is already hard linked to one of the backends (presumably libhidapi-libusb in your case), and there is no "official" way to change it.

Be-ing commented 3 years ago

I hacked the application build system to use hidraw instead of libusb. Now it fails to open the device, both with hid_open_path and hid_open without any error message from hidapi indicating the problem...

debug [Controller] CDBG Opening HID device Traktor Kontrol S4 MK3 A7A8_3 by HID path /dev/hidraw3
debug [Controller] CDBG Failed. Trying to open with make, model & serial no: 6092 5920 69DDA7A8
warning [Controller] Unable to open specific HID device "Traktor Kontrol S4 MK3 A7A8_3" Trying now with just make and model. (This may only open the first of multiple identical devices.)
warning [Controller] Unable to open HID device "Traktor Kontrol S4 MK3 A7A8_3"
debug [Controller] Controller polling stopped.

Here is the application code.

Be-ing commented 3 years ago

I'm confused why the manufacturer set the wMaxPacketSize for the HID endpoint to 64 when they made the HID protocol have packets larger than 64 byes. I'm also confused how the packets are not split on macOS and Windows... do those operating systems concatenate the packet fragments in their HID drivers??

@JoergAtGithub what makes you think that the devices are opened as Full-Speed USB 1 devices? Is it only that the packets are split at 64 bytes or is there some other hint?

JoergAtGithub commented 3 years ago

@JoergAtGithub what makes you think that the devices are opened as Full-Speed USB 1 devices? Is it only that the packets are split at 64 bytes or is there some other hint?

One of the Linux users reported this:

lsusb -D /dev/bus/usb/005/006

Device: ID 17cc:1723 Native Instruments Traktor Kontrol S4 MK3 Hub
Device Descriptor:
  bLength                18
  bDescriptorType         1
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 
  bDeviceProtocol         2 TT per port
  bMaxPacketSize0        64
  idVendor           0x17cc Native Instruments
  idProduct          0x1723 
  bcdDevice            0.01
  iManufacturer           1 Native Instruments
  iProduct                2 Traktor Kontrol S4 MK3 Hub
  iSerial                 3 7AB2CCAD
  bNumConfigurations      1
  Configuration Descriptor:
    bLength                 9
    bDescriptorType         2
    wTotalLength       0x0029
    bNumInterfaces          1
    bConfigurationValue     1
    iConfiguration          0 
    bmAttributes         0xe0
      Self Powered
      Remote Wakeup
    MaxPower                0mA
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       0
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 
      bInterfaceProtocol      1 Single TT
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
    Interface Descriptor:
      bLength                 9
      bDescriptorType         4
      bInterfaceNumber        0
      bAlternateSetting       1
      bNumEndpoints           1
      bInterfaceClass         9 Hub
      bInterfaceSubClass      0 
      bInterfaceProtocol      2 TT per port
      iInterface              0 
      Endpoint Descriptor:
        bLength                 7
        bDescriptorType         5
        bEndpointAddress     0x81  EP 1 IN
        bmAttributes            3
          Transfer Type            Interrupt
          Synch Type               None
          Usage Type               Data
        wMaxPacketSize     0x0001  1x 1 bytes
        bInterval              12
Hub Descriptor:
  bLength               9
  bDescriptorType      41
  nNbrPorts             2
  wHubCharacteristic 0x0009
    Per-port power switching
    Per-port overcurrent protection
    TT think time 8 FS bits
  bPwrOn2PwrGood        0 * 2 milli seconds
  bHubContrCurrent      0 milli Ampere
  DeviceRemovable    0x00
  PortPwrCtrlMask    0xff
 Hub Port Status:
   Port 1: 0000.0503 highspeed power enable connect
   Port 2: 0000.0100 power
Device Qualifier (for other device speed):
  bLength                10
  bDescriptorType         6
  bcdUSB               2.00
  bDeviceClass            9 Hub
  bDeviceSubClass         0 
  bDeviceProtocol         0 Full speed (or root) hub
  bMaxPacketSize0        64
  bNumConfigurations      1
can't get debug descriptor: Resource temporarily unavailable
Device Status:     0x0001
  Self Powered

AFAIK the slowest element in the USB bus chain defines the speed. I assumed, that the HUB limits the speed of the HID endpoint to Full-Speed.

Be-ing commented 3 years ago

The hub operates at 480 Mb too. Here is lsusb -v with a USB 3.0 drive plugged into the USB-A port on the device:

    |__ Port 3: Dev 5, If 0, Class=Hub, Driver=hub/2p, 480M
        |__ Port 2: Dev 8, If 0, Class=Mass Storage, Driver=usb-storage, 480M
        |__ Port 1: Dev 6, If 5, Class=Application Specific Interface, Driver=, 480M
        |__ Port 1: Dev 6, If 3, Class=Human Interface Device, Driver=usbhid, 480M
        |__ Port 1: Dev 6, If 1, Class=Audio, Driver=snd-usb-audio, 480M
        |__ Port 1: Dev 6, If 4, Class=Vendor Specific Class, Driver=, 480M
        |__ Port 1: Dev 6, If 2, Class=Audio, Driver=snd-usb-audio, 480M
        |__ Port 1: Dev 6, If 0, Class=Audio, Driver=snd-usb-audio, 480M
Be-ing commented 3 years ago

I don't think the USB speed is the problem here. I think it is the wMaxPacketSize of 64.

Youw commented 3 years ago

I'm also confused how the packets are not split on macOS and Windows... do those operating systems concatenate the packet fragments in their HID drivers??

I wouldn't be suprised.

Be-ing commented 3 years ago

Does the HID specification say to concatenate the packet fragments in this case with wMaxPacketSize smaller than the report size? I wouldn't be surprised if this is an edge case where the specification does not say what to do... I'm still at a loss why the manufacturer would create this situation.

Be-ing commented 3 years ago

The HID specification has this to say about wMaxPacketSize:

All reports except the longest which exceed wMaxPacketSize for the endpoint must terminate with a short packet. The longest report does not require a short packet terminator.

I'm unclear exactly what that means but it seems that it permits wMaxPacketSize to be shorter than the report size.

Be-ing commented 3 years ago

I'm also confused how the packets are not split on macOS and Windows... do those operating systems concatenate the packet fragments in their HID drivers??

This would make some sense. libusb by itself has no awareness that the split USB packets should be concatenated for the HID protocol. However hidapi can determine this.

Be-ing commented 3 years ago

hidapi can determine this.

hidapi would need to parse the report descriptor to determine this, so I guess #249 is a prerequisite for this?

JoergAtGithub commented 3 years ago

Isn't the report_descriptor already available internal in the libusb backend: https://github.com/libusb/hidapi/blob/b72a3675551a233e071ebf61e3af22642ea9ce0c/libusb/hid.c#L643

Be-ing commented 3 years ago

Yes, getting the report descriptor is there. Parsing it is another task. hidapi would need to compare the size of the reports described in the report descriptor to wMaxPacketSize. If any reports are bigger than wMaxPacketSize, keep incoming packets that are equal in length to wMaxPacketSize in a buffer, then concatenate the buffer with the short part when the incoming packet length is equal to the long report length minus wMaxPacketSize.

Be-ing commented 3 years ago

The quick and ugly hack around this is for the application to concatenate the packets for devices know to be affected by this, but that means the application requires platform-specific code to use a cross-platform library. :/

Youw commented 3 years ago

Advanced parsing and packet (re-)construction/verification was never the intention of hidapi. It always tried to be the thinest proxy library possible, with remarks to keeping the API uniform across platforms.

In defence of the application side workaround for a specific device - I believe the reconstruction code can be written uniformly for all platforms, it just that some of the branches of the code will be executed when run on linux only.

Be-ing commented 3 years ago

Advanced parsing and packet (re-)construction/verification was never the intention of hidapi. It always tried to be the thinest proxy library possible, with remarks to keeping the API uniform across platforms.

This issue makes the API not uniform across platforms.

In defence of the application side workaround for a specific device - I believe the reconstruction code can be written uniformly for all platforms, it just that some of the branches of the code will be executed when run on linux only.

This could not be done in a generic way for any device because that would require hidapi to expose the wMaxPacketSize -- or possibly directly invoke libusb to get that, which kinda defeats the point of using hidapi. For the few devices known to be affected by this for Mixxx, it isn't hard to hack around this in the device-specific JavaScript code that maps the HID I/O to application logic.

JoergAtGithub commented 3 years ago

I don't think, that it's neccessary to have wMaxPacketSize to implement this. The only information you need is, the size of each report by ReportID. And this information can be gathered from the report descriptor. With this information you could check if the report is complete, or if another data packet needs do be received.

Be-ing commented 3 years ago

I got the hidraw backend working by adding a new udev rule. The hidraw backend does not split the big report. The application receives a single 79 byte report like on Windows and macOS. I did not realize the libusb and hidraw backends require different udev rules. I think linking to the hidraw backend would be an acceptable workaround.

Be-ing commented 3 years ago

I don't think, that it's neccessary to have wMaxPacketSize to implement this. The only information you need is, the size of each report by ReportID. And this information can be gathered from the report descriptor. With this information you could check if the report is complete, or if another data packet needs do be received.

That would require hidapi (or the application) to do some hacky guesswork. If the incoming packet is not equal to the size of a known report from the report descriptor, store it in a buffer. Then, when another incoming packet is another size that isn't in the size of a known report descriptor, concatenate it with the buffer.

JoergAtGithub commented 3 years ago

Not any packet of unknown size - you have to read the next packet until the report size is reached, and than you return the concatenated report to the user.

Be-ing commented 3 years ago

Okay, that seems considerably simpler to implement.

JoergAtGithub commented 3 years ago

Since you proved, that the hidraw Linux kernel module handles this case correct, there is at least one working OpenSource implementation available, which can be taken as reference to fix HIDAPIs libusb backend implementation.

Be-ing commented 3 years ago

Here is Linux's hidraw driver's implementation of hidraw_read.

Youw commented 3 years ago

Don't see anything regarding concatination there, even remotely.

Be-ing commented 3 years ago

I guess the concatenation happens at a lower layer of the kernel, presumably somewhere in drivers/hid/hid-core.c. The common entry point of the different wire protocols is hid_input_report. Or maybe it happens somewhere in drivers/hid/usbhid/hid-core.c.

mcuee commented 3 years ago

It seems that libusb can only read a device's speed with libusb_get_device_speed() but has no API to set the speed.

It is the kernel driver's job. libusb is a user mode library.

mcuee commented 3 years ago

Ref: https://community.arm.com/developer/tools-software/tools/f/keil-forum/26534/small-question-about-hid-usb-device

wMaxPacketSize of the interrupt IN (or OUT) endpoint determines the division unit of the report. For full-speed device, wMaxPacketSize is limited to less than or equal to 64 bytes. For low-speed, it's 8 bytes.

When the report size is greater than wMaxPacketSize, the report is split into packets of wMaxPacketSize. For example, 128 bytes report is divided into two 64 bytes packets. And then, these packets are exchanged one packet by one over the interrupt endpoint. I'm not sure which HID example you are starting on. In most examples, you have to code it so that above division works.

bMaxPacketSize of the default endpoint (EP0) also determines the division unit of the transfer. For HID reports, this value affects to the report exchange using Get_Report and Set_Report() requests. Usually in most examples, the USB firmware stack cares it for you.

Tsuneo

mcuee commented 3 years ago

Technically libusb backend should not need to worry about individual packet but rather the transfer. The transfer size should be of the report size. The data on the wire is based on packet, but then the transfer can split into multiple packets.

This is probably what happens under the hidraw backend, as well as the Windows and macOS backend using OS native HID driver. I am not so sure why the hidapi libusb backend is different in this case.

mcuee commented 3 years ago

Most of the HID device will have the wMaxPacketSize as 8 bytes (low speed) or 64 bytes (full speed). I remember seeing old reports that Windows default HID driver (at least for older version of Windows) does not support high speed interrupt endpoints. That does limit the speed of HID device. However, it does not limit the report size.

mcuee commented 3 years ago

https://github.com/libusb/libusb/blob/master/libusb/os/windows_winusb.c#L3728 static int hid_open(int sub_api, struct libusb_device_handle *dev_handle)

The libusb Windows HID backend has some codes to figure out the report size.

        // Set the report sizes
        priv->hid->input_report_size = capabilities.InputReportByteLength;
        priv->hid->output_report_size = capabilities.OutputReportByteLength;
        priv->hid->feature_report_size = capabilities.FeatureReportByteLength;

        // Store usage and usagePage values
        priv->hid->usage = capabilities.Usage;
        priv->hid->usagePage = capabilities.UsagePage;
Youw commented 3 years ago

Same way it is done by hidapi on Windows backend. The problem here is on Linux with libusb backend for hidapi

mcuee commented 3 years ago

For Linux libusb backend of HIDAPI, can we do the same thing? Figure out the report size, and then use Control Transfer or Interrupt transfer to request for the report size. Then the Linux usbfs driver will handle the split of packet.

Am I missing something obvious?

Youw commented 3 years ago

I believe we can parse hid descriptor (which we already doing for some other purposes) and find out the largest report size, and use it as a length parameter passed to libusb_fill_interrupt_transfer.

Now some questions:

Youw commented 3 years ago

This is probably what happens under the hidraw backend, as well as the Windows and macOS backend using OS native HID driver.

I am not so sure why the hidapi libusb backend is different in this case.

Because in all cases other than libusb - the backend is an OS/Driver implementation. hidapi just uses it thru an API. Libusb backend is basically an alternative implementation of such "driver" written by Alan, Ludovic and a few other folks from the community. libusb backend is the only backend that implements a low-level USB/HID protocol.

mcuee commented 3 years ago

Yes I agree that we are implementing the USB/HID protocol with libusb, however, I am not so convinced about the packet splitting thingy.

Because in all cases other than libusb - the backend is an OS/Driver implementation. hidapi just uses it thru an API. Libusb backend is basically an alternative implementation of such "driver" written by Alan, Ludovic and a few other folks from the community. libusb backend is the only backend that implements a low-level USB/HID protocol.

mcuee commented 3 years ago

I believe we can parse hid descriptor (which we already doing for some other purposes) and find out the largest report size, and use it as a length parameter passed to libusb_fill_interrupt_transfer.

Now some questions:

  • will libusb (or whatever backend there is) automatically concatenate several parts of the report?
  • what if we have a small report which is at least 2 times smaller than the largest report - is there a chance that two or more reports will be contatenated into a single large buffer passed to transfer?

I believe the answers are. 1) YES 2) NO

But I actually do not know the internal of libusb very well. I will ask in the libusb mailing list.

Youw commented 3 years ago

I believe the answers are.

  1. YES
  2. NO

That would be the most desirable behavior from this issue/feature POV.