Some hardware testing, mixed results

zhogov commented 4 years ago

Thanks for this great tool! I wanted use it for switching my keyboard/mouse between my work Macbook and home PC. So I tested it in the following setup:

Raspberry Pi Zero W + Apple Magic Keyboard A1644 + Tecknet M003 mouse

Worked perfectly on:

No mouse, but I'm not sure if mouse should've worked on phones

iPhone 11
iPad 6th generation
LG G6 (android smartphone)

Windows 10:

laptop, built-in Bluetooth
tower, USB Bluetooth adapter

Problems:

very noticeable keyboard lag
if I quickly press two keys simultaneously, say "g" and "h" then one of them will work and another will get stuck for about 5 seconds: ghhhhhhhhhhh
no mouse

Mac OS on MacBook Pro (15-inch, 2019)

Problems:

Bluetooth connection happens, but no keypresses are working. Maybe Mac doesn't even recognize it as keyboard?
no mouse

Ubuntu 20.04 on Intel NUC 7i3BNK

Problems:

Bluetooth connection happens, but no keypresses are working.
no mouse

zhogov commented 4 years ago

How could I troubleshoot issues on Win10 and Mac?

ruundii commented 4 years ago

Thanks for this. do you mind sharing how you configured the devices on web UI? I have not had a chance yet to test keyboard lags with the compatibility mode on RPi Zero (on holiday and don't have one around). Compatibility mode lag will always be higher than the direct hidraw mode with a proper filter (as essentially in the hidraw mode I take the hid reports straight away, do very simple filter processing and route those to a received. In a compatibility mode the reports get full OS processing and only after that I get the events and send those as hid reports to a receiver).

There was another source of latency I had when was developing the app - Bluetooth. There were occasional periods of lags, particularly when there were several hosts connected (even though I send to only one host at a time). When I investigated I realised that if the Raspberry pi is in Bluetooth slave mode (the way Bluetooth piconets are designed) it is basically waiting for too long until Bluetooth master(s) tell the raspberry it can send the Bluetooth packets. I dealt with it by setting raspberry to master mode for all its connections over here. Maybe still worth checking if it actually works on your setup and that raspberry in master mode (run 'sudo hcitool con' to see status of the connections).

As for other set ups, are you trying them one at the time or they are all hosts connected at the same time? It may be that it's just not active hosts? (currently host switching is implemented in A1314 keyboard filter over here. I probably need to generalise the host switching to other keyboards, which I will be able to do in few days. But for now to test it you can just make sure you have only one host connected (or paired) at the time, so the hub will surely be sending hid reports to this host and not another.

ruundii commented 4 years ago

Also, looking at your screenshot from issue#4, I see you use the default filter. I am now contemplating how I can improve troubleshooting/tracing/choosing a right configuration for a wide range of devices in the next version of the app. The default filter basically sends hid reports as is. it will work only if the device hid descriptor is more or less the same as the one I use for BT HID Hub (as the host machines see BT HID Hub). All the fancy devices, combo devices etc would have their hid descriptors quite different from the basic one I use. So those would require a custom filter, or need to work in a compatibility mode (possibly with a lag, I have not investigated if there is lag because of the compatibility mode). possibly over time I can develop functionality to configure a custom filter / hid report mapping, rather than requiring to develop it with a code. Or, alternatively, will develop some good hid report tracing functionality, so it will be really easy to develop a custom hid filter without going into a depth of the codebase.

Would be interesting to see the hid descriptor of your K400 Plus. could you pull one, using hid-tools?

Thanks

zhogov commented 4 years ago

All devices were tested one-by one, here are the images:

zhogov commented 4 years ago

While I was taking screenshots I found a compatibility mode that I somehow missed. It was disabled. After enabling compatibility mode the keyboard started to work on Mac and Windows but not Ubuntu, (while still working on iPhone).

I don't get it though – why did it start working?

In compatibility mode BT HID Hub will capture the keyboard events via EVDEV after they are processed by OS and will create a virtual HID Raw keyboard as a destination for those events.

zhogov commented 4 years ago

The default filter basically sends hid reports as is. it will work only if the device hid descriptor is more or less the same as the one I use for BT HID Hub (as the host machines see BT HID Hub).

So after reading the nice article about HID descriptors you suggested I start to understand why it worked with compat mode (or more like why it didn't with the Default filter)

After that I tried using my Apple Magic Keyboard A1644 with A1314 filter – idea was that Apple devices could work similarly – but without luck.

zhogov commented 4 years ago

Thinking more about supporting different devices in more generic way:

Filters take custom reports and turn them into standard/generic/common bthidhub report that is sent over BT. Parsing these reports for every device seems to be pretty complicated task, eventually we will end up with code that is equivalent to Linux HID parser, right? Doesn't look like a good long-term strategy.
Compatibility mode is great – we reuse parts of Linux that try to make sense of reports and also handle bugs of some specific devices, but
- it is less flexible as working with HID Raw devices, as some vendor-specific key (e.g. fn button) may only be captured before OS processing (HID RAW), not after (via EVDEV)
- also I'm a bit worried about Linux on Raspberry parsing HID events differently from what would Mac or Windows do
- the latency is a concern, how can I test it?
Could it be a good idea to completely clone HID descriptor of a device we want to multiplex and just pass-through the reports with some minimal user-defined remapping?
- Hosts will see a correct descriptor and reports matching this descriptor
- Remapping is still possible and is more flexible than compatibility mode
- Linux won't have to parse these events making system less complex
- I'm not sure how this works with multiple devices though. I think that this can be solved by using Application Collection – create collection of cloned descriptors for each input device
```
-> Report
-> Application Collection
-> Mouse Descriptor
-> Application Collection
-> Mouse Descriptor
```
  An Application Collection describes a set of inputs that make sense as a whole. Usually, every Report Descriptor must define at least one Application Collection but you may have two or more. For example, a a keyboard with integrated trackpoint should and/or would use two. This is how the kernel knows it needs to create two separate event nodes for the device.

zhogov commented 4 years ago

As for Logitech K400 Plus descriptor: /sys/class/input/event17/device/device/uevent

DRIVER=logitech-hidpp-device
HID_ID=0003:0000046D:0000404D
HID_NAME=Logitech K400 Plus
HID_PHYS=usb-0000:00:14.0-1/input2:1
HID_UNIQ=404d-62-63-dc-da
MODALIAS=hid:b0003g0102v0000046Dp0000404D

./hid-decode /sys/class/input/event17/device/device/report_descriptor

# device 0:0
# 0x05, 0x01,                    // Usage Page (Generic Desktop)        0
# 0x09, 0x06,                    // Usage (Keyboard)                    2
# 0xa1, 0x01,                    // Collection (Application)            4
# 0x85, 0x01,                    //  Report ID (1)                      6
# 0x95, 0x08,                    //  Report Count (8)                   8
# 0x75, 0x01,                    //  Report Size (1)                    10
# 0x15, 0x00,                    //  Logical Minimum (0)                12
# 0x25, 0x01,                    //  Logical Maximum (1)                14
# 0x05, 0x07,                    //  Usage Page (Keyboard)              16
# 0x19, 0xe0,                    //  Usage Minimum (224)                18
# 0x29, 0xe7,                    //  Usage Maximum (231)                20
# 0x81, 0x02,                    //  Input (Data,Var,Abs)               22
# 0x95, 0x06,                    //  Report Count (6)                   24
# 0x75, 0x08,                    //  Report Size (8)                    26
# 0x15, 0x00,                    //  Logical Minimum (0)                28
# 0x26, 0xff, 0x00,              //  Logical Maximum (255)              30
# 0x05, 0x07,                    //  Usage Page (Keyboard)              33
# 0x19, 0x00,                    //  Usage Minimum (0)                  35
# 0x2a, 0xff, 0x00,              //  Usage Maximum (255)                37
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               40
# 0x85, 0x0e,                    //  Report ID (14)                     42
# 0x05, 0x08,                    //  Usage Page (LEDs)                  44
# 0x95, 0x05,                    //  Report Count (5)                   46
# 0x75, 0x01,                    //  Report Size (1)                    48
# 0x15, 0x00,                    //  Logical Minimum (0)                50
# 0x25, 0x01,                    //  Logical Maximum (1)                52
# 0x19, 0x01,                    //  Usage Minimum (1)                  54
# 0x29, 0x05,                    //  Usage Maximum (5)                  56
# 0x91, 0x02,                    //  Output (Data,Var,Abs)              58
# 0x95, 0x01,                    //  Report Count (1)                   60
# 0x75, 0x03,                    //  Report Size (3)                    62
# 0x91, 0x01,                    //  Output (Cnst,Arr,Abs)              64
# 0xc0,                          // End Collection                      66
# 0x05, 0x01,                    // Usage Page (Generic Desktop)        67
# 0x09, 0x02,                    // Usage (Mouse)                       69
# 0xa1, 0x01,                    // Collection (Application)            71
# 0x85, 0x02,                    //  Report ID (2)                      73
# 0x09, 0x01,                    //  Usage (Pointer)                    75
# 0xa1, 0x00,                    //  Collection (Physical)              77
# 0x05, 0x09,                    //   Usage Page (Button)               79
# 0x19, 0x01,                    //   Usage Minimum (1)                 81
# 0x29, 0x10,                    //   Usage Maximum (16)                83
# 0x15, 0x00,                    //   Logical Minimum (0)               85
# 0x25, 0x01,                    //   Logical Maximum (1)               87
# 0x95, 0x10,                    //   Report Count (16)                 89
# 0x75, 0x01,                    //   Report Size (1)                   91
# 0x81, 0x02,                    //   Input (Data,Var,Abs)              93
# 0x05, 0x01,                    //   Usage Page (Generic Desktop)      95
# 0x16, 0x01, 0xf8,              //   Logical Minimum (-2047)           97
# 0x26, 0xff, 0x07,              //   Logical Maximum (2047)            100
# 0x75, 0x0c,                    //   Report Size (12)                  103
# 0x95, 0x02,                    //   Report Count (2)                  105
# 0x09, 0x30,                    //   Usage (X)                         107
# 0x09, 0x31,                    //   Usage (Y)                         109
# 0x81, 0x06,                    //   Input (Data,Var,Rel)              111
# 0x15, 0x81,                    //   Logical Minimum (-127)            113
# 0x25, 0x7f,                    //   Logical Maximum (127)             115
# 0x75, 0x08,                    //   Report Size (8)                   117
# 0x95, 0x01,                    //   Report Count (1)                  119
# 0x09, 0x38,                    //   Usage (Wheel)                     121
# 0x81, 0x06,                    //   Input (Data,Var,Rel)              123
# 0x05, 0x0c,                    //   Usage Page (Consumer Devices)     125
# 0x0a, 0x38, 0x02,              //   Usage (AC Pan)                    127
# 0x95, 0x01,                    //   Report Count (1)                  130
# 0x81, 0x06,                    //   Input (Data,Var,Rel)              132
# 0xc0,                          //  End Collection                     134
# 0xc0,                          // End Collection                      135
# 0x05, 0x0c,                    // Usage Page (Consumer Devices)       136
# 0x09, 0x01,                    // Usage (Consumer Control)            138
# 0xa1, 0x01,                    // Collection (Application)            140
# 0x85, 0x03,                    //  Report ID (3)                      142
# 0x75, 0x10,                    //  Report Size (16)                   144
# 0x95, 0x02,                    //  Report Count (2)                   146
# 0x15, 0x01,                    //  Logical Minimum (1)                148
# 0x26, 0xff, 0x02,              //  Logical Maximum (767)              150
# 0x19, 0x01,                    //  Usage Minimum (1)                  153
# 0x2a, 0xff, 0x02,              //  Usage Maximum (767)                155
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               158
# 0xc0,                          // End Collection                      160
# 0x05, 0x01,                    // Usage Page (Generic Desktop)        161
# 0x09, 0x80,                    // Usage (System Control)              163
# 0xa1, 0x01,                    // Collection (Application)            165
# 0x85, 0x04,                    //  Report ID (4)                      167
# 0x75, 0x02,                    //  Report Size (2)                    169
# 0x95, 0x01,                    //  Report Count (1)                   171
# 0x15, 0x01,                    //  Logical Minimum (1)                173
# 0x25, 0x03,                    //  Logical Maximum (3)                175
# 0x09, 0x82,                    //  Usage (System Sleep)               177
# 0x09, 0x81,                    //  Usage (System Power Down)          179
# 0x09, 0x83,                    //  Usage (System Wake Up)             181
# 0x81, 0x60,                    //  Input (Data,Arr,Abs,NoPref,Null)   183
# 0x75, 0x06,                    //  Report Size (6)                    185
# 0x81, 0x03,                    //  Input (Cnst,Var,Abs)               187
# 0xc0,                          // End Collection                      189
# 0x06, 0x00, 0xff,              // Usage Page (Vendor Defined Page 1)  190
# 0x09, 0x01,                    // Usage (Vendor Usage 1)              193
# 0xa1, 0x01,                    // Collection (Application)            195
# 0x85, 0x10,                    //  Report ID (16)                     197
# 0x75, 0x08,                    //  Report Size (8)                    199
# 0x95, 0x06,                    //  Report Count (6)                   201
# 0x15, 0x00,                    //  Logical Minimum (0)                203
# 0x26, 0xff, 0x00,              //  Logical Maximum (255)              205
# 0x09, 0x01,                    //  Usage (Vendor Usage 1)             208
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               210
# 0x09, 0x01,                    //  Usage (Vendor Usage 1)             212
# 0x91, 0x00,                    //  Output (Data,Arr,Abs)              214
# 0xc0,                          // End Collection                      216
# 0x06, 0x00, 0xff,              // Usage Page (Vendor Defined Page 1)  217
# 0x09, 0x02,                    // Usage (Vendor Usage 2)              220
# 0xa1, 0x01,                    // Collection (Application)            222
# 0x85, 0x11,                    //  Report ID (17)                     224
# 0x75, 0x08,                    //  Report Size (8)                    226
# 0x95, 0x13,                    //  Report Count (19)                  228
# 0x15, 0x00,                    //  Logical Minimum (0)                230
# 0x26, 0xff, 0x00,              //  Logical Maximum (255)              232
# 0x09, 0x02,                    //  Usage (Vendor Usage 2)             235
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               237
# 0x09, 0x02,                    //  Usage (Vendor Usage 2)             239
# 0x91, 0x00,                    //  Output (Data,Arr,Abs)              241
# 0xc0,                          // End Collection                      243
# 0x06, 0x00, 0xff,              // Usage Page (Vendor Defined Page 1)  244
# 0x09, 0x04,                    // Usage (Vendor Usage 0x04)           247
# 0xa1, 0x01,                    // Collection (Application)            249
# 0x85, 0x20,                    //  Report ID (32)                     251
# 0x75, 0x08,                    //  Report Size (8)                    253
# 0x95, 0x0e,                    //  Report Count (14)                  255
# 0x15, 0x00,                    //  Logical Minimum (0)                257
# 0x26, 0xff, 0x00,              //  Logical Maximum (255)              259
# 0x09, 0x41,                    //  Usage (Vendor Usage 0x41)          262
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               264
# 0x09, 0x41,                    //  Usage (Vendor Usage 0x41)          266
# 0x91, 0x00,                    //  Output (Data,Arr,Abs)              268
# 0x85, 0x21,                    //  Report ID (33)                     270
# 0x95, 0x1f,                    //  Report Count (31)                  272
# 0x15, 0x00,                    //  Logical Minimum (0)                274
# 0x26, 0xff, 0x00,              //  Logical Maximum (255)              276
# 0x09, 0x42,                    //  Usage (Vendor Usage 0x42)          279
# 0x81, 0x00,                    //  Input (Data,Arr,Abs)               281
# 0x09, 0x42,                    //  Usage (Vendor Usage 0x42)          283
# 0x91, 0x00,                    //  Output (Data,Arr,Abs)              285
# 0xc0,                          // End Collection                      287
# 
R: 288 05 01 09 06 a1 01 85 01 95 08 75 01 15 00 25 01 05 07 19 e0 29 e7 81 02 95 06 75 08 15 00 26 ff 00 05 07 19 00 2a ff 00 81 00 85 0e 05 08 95 05 75 01 15 00 25 01 19 01 29 05 91 02 95 01 75 03 91 01 c0 05 01 09 02 a1 01 85 02 09 01 a1 00 05 09 19 01 29 10 15 00 25 01 95 10 75 01 81 02 05 01 16 01 f8 26 ff 07 75 0c 95 02 09 30 09 31 81 06 15 81 25 7f 75 08 95 01 09 38 81 06 05 0c 0a 38 02 95 01 81 06 c0 c0 05 0c 09 01 a1 01 85 03 75 10 95 02 15 01 26 ff 02 19 01 2a ff 02 81 00 c0 05 01 09 80 a1 01 85 04 75 02 95 01 15 01 25 03 09 82 09 81 09 83 81 60 75 06 81 03 c0 06 00 ff 09 01 a1 01 85 10 75 08 95 06 15 00 26 ff 00 09 01 81 00 09 01 91 00 c0 06 00 ff 09 02 a1 01 85 11 75 08 95 13 15 00 26 ff 00 09 02 81 00 09 02 91 00 c0 06 00 ff 09 04 a1 01 85 20 75 08 95 0e 15 00 26 ff 00 09 41 81 00 09 41 91 00 85 21 95 1f 15 00 26 ff 00 09 42 81 00 09 42 91 00 c0
N: device 0:0
I: 3 0001 0001

ruundii commented 4 years ago

Indeed, a good summary of options. Initially, I went with an option 1, not the least because this is what I needed for my A1314 keyboard. I researched and seen quite a bit of people having the same issue (key remapping) with apple keyboards, so this class of problems is addressable with a few device-specific filters (in future maybe need to develop an easy ability for people to customise the re-mappings though).

option 2 - latency testing - good question. I reckon that by using evdev I am not losing an ability to get read hidraw. Possibly just trace timestamps of the events as they come to hidraw vs when you get evdev event. Or just create a hidraw filter for the same device and compare how it feels. As I mentioned before, I have done some tracing with hidraw filter-based implementation before. I remember it has been single milliseconds from the time hidraw file is read to the time Bluetooth packet is pushed to the socket in bluez (if in Bluetooth master mode).

the third option is something I also have been thinking of. there are couple of possible issues on this path: 1) Bluetooth hid descriptor is published in the sdp record and is read only on pairing. so, any changes will require re-pairing. maybe not a significant issue, but certainly something to be aware of. e.g. if user brings a new keyboard, they would need to configure it in the hub and re-pair. 2) indeed, multi-device case. I now combine keyboard and mouse descriptors. theoretically, it may be possible to develop some code to merge the descriptors and cover hid report id mapping. Maybe need to explore it further. 3) there still may be a limitation related to vendor-specific drivers. my understanding is that OS will apply a vendor-specific driver based on vendor/product id of the device. I believe these can be set in the Bluetooth main.conf file, but then I guess host OS may have a USB driver for such vendor/product id, but not a Bluetooth one. Also, if there are two vendor-specific devices it will not work. At the same time, this limitation is still present in the current implementation, as there is no out-of-box support for vendor-specific matters - so, not losing anything, can try. Based on previous point though, I would go with an attempt to automatically merge descriptors and remap report ids.

ruundii commented 4 years ago

I have now done my first subjective testing of the compatibility mode on raspberry pi zero + windows as a host. And I cannot say I notice any lag or difference with the hidraw mode.

zhogov commented 4 years ago

Yeah, I was running compat mode with keyboard only for two days now, works ok. Even for gaming

cnnblike commented 3 years ago

similar issue, latency around few seconds and totally not usable, maybe I will take some time to troubleshoot this or just implement something totally different from this (guess a implementation of OS-less version would be more handy.)

Dreamsorcerer commented 3 years ago

Could someone with the latency issue confirm if this is still an issue with the current code? If you ssh into the RPi, you should be able to update with:

cd bthidhub/
git pull
sudo service remapper restart

I've fixed an issue with it not going back to master mode when reconnecting to the host, so this may be the same issue that others are reporting.

Dreamsorcerer commented 3 years ago

Going to close this if nobody else can reproduce. I think I've covered the other issues in #23 and maybe #21.

ruundii / bthidhub