Pico keyboard is not "quite" a real keyboard?

[callimero]

I build a external keyboard with a Pico and the adafruit hid.

Using Win 10. It works well in the focussed app. BUT only sends(?) to foreground apps. I discovered this in OBS Studio, I can teach in the keystrokes my little Pico-Keyboard sends to OBS in the Hotkey section. And they work. The DO not work when I focus the App I want to record in OBS.

A test with a second just plain normal USB keyboard does not show that! So It seems the Pico-HID is some different kind. Maybe a security Reason? (e.g. if you make a malicious USB dongle which types in bad commands?)

[ladyada]

what are the exact key-codes you are sending and key-strokes that work

[dhalbert]

OBS responds to both regular-keyboard shortcuts for things like volume control, stop, and start, and to "Multimedia keys", which are actually sent on a separate HID device called "Consumer Control". Your regular physical keyboard is probably presenting as both. The multimedia keys on it (record, stop, start, volume up/down, etc.) are sent on the consumer control device.

Consumer control doesn't require keyboard focus. So you can add adafruit_hid.consumer_control.ConsumerControl sends to your Pico keyboard and it should work.

The confusion is that OBS responds to both, and has regular keyboard shortcuts that overlap with the consumer control keys.

[ladyada]

yah thats what im thinking ... @kattni plz add a note to our HID pico guides & when you template our essentials page for HID plz add a note there too!

[kattni]

@ladyada A note has been added to the only Pico HID guide I could find. https://learn.adafruit.com/diy-pico-mechanical-keyboard-with-fritzing-circuitpython/code-the-pico-keyboard#usb-keyboards-versus-circuitpython-hid-keyboards-3084311-9

I have also made a note to include it in the HID template when I get to it.

[callimero]

Wow, great feedback. Just preparing a article about the matter (using Pico as Keyboard) .

The thing is that a second USB keyboard together with my usual USB keyboard (Laptop has also a PS/2 bound keyboard but is closed) sends the exact same key combinations to "unfocussed" OBS just fine.

I tried several combinations, also some which no modern app knows like [Keycode.ALT, Keycode.CONTROL, Keycode.F14]

Oh and btw: my Arduino Leonardo based design (2014) of that matter has no problem sending Keys to a unfocussed app... https://forum.arduino.cc/t/foot-switch-box-for-rocksmith-and-other-applications/149182

I am not using CC commands in the simple form I tried. It looks like this:

import board
import digitalio
import time
import usb_hid
from adafruit_hid.keyboard import Keyboard
from adafruit_hid.keycode import Keycode

taste = digitalio.DigitalInOut(board.GP16) # GPIO 16
taste.switch_to_input(pull=digitalio.Pull.DOWN) # int. Pulldown

kbd = Keyboard(usb_hid.devices)

while True:
    if taste.value:
        kbd.send(Keycode.SHIFT,Keycode.ALT,Keycode.LEFT_CONTROL,Keycode.K)    
        time.sleep(0.4)       

[dhalbert]

The thing is that a second USB keyboard together with my usual USB keyboard (Laptop has also a PS/2 bound keyboard but is closed) sends the exact same key combinations to "unfocussed" OBS just fine.

I'm not sure I understand the distinction about the second keyboard. So you are saying that the key combinations from the first or second keyboard are not Consumer Control?

The other difference between CircuitPython Keyboard and a typical physical keyboard is that the physical keyboard usually presents as a non-composite HID keyboard, as a "BOOT" keyboard, so that you can use it to talk to a BIOS, etc. The CircuitPython keyboard is a non-BOOT keyboard. It's possible that either OBS or the OS is somehow noticing this difference, but I'm not sure why.

We are adding dynamic USB descriptors to CircuitPython 7.0.0, including user-supplied HID descriptors and the ability to turn other devices on and off (e.g. turn off CIRCUITPY). I haven't included boot keyboards in the first pass of this addition but may do it later.

[callimero]

I'm not sure I understand the distinction about the second keyboard. So you are saying that the key combinations from the first or second keyboard are not Consumer Control?

So far I am using NO CC at all. I just brought in a second keyboard as datapoint, that it is not a genereal fault with more than one keyboard. Same for my arduino box.

In the meantime I also tested on another laptop with Windows 8.1 and the adafruit-HID has the same problem there.

The other difference between CircuitPython Keyboard and a typical physical keyboard is that the physical keyboard usually presents as a non-composite HID keyboard, as a "BOOT" keyboard, so that you can use it to talk to a BIOS, etc. The CircuitPython keyboard is a non-BOOT keyboard. It's possible that either OBS or the OS is somehow noticing this difference, but I'm not sure why.

That BOOT may be the problem. So I can imagine at least two use cases for that which make it useful to have in CircuitPython: There are applications with PCs where you sometimes need some keys but a full keyboard is not possible (Booting, skipping errors, controlling boot process) and of course controlling apps in the back like streaming clients, video-chats, PTT in games etc.

EDIT: Maybe the Arduino HID is some place to look at. At least in the 2013 Version I used in my Box then it worked, but that is a Arduino Micro which has a special USB chip. Not sure about the "software hid".

So thanks for your support, please consider this as a useful addtion. I see fancy streamdecks coming with the power of RP2040 boards.

[LostVector]

Is this issue (boot keyboard vs standard usb composite device) why I can't wake a sleeping computer with neokey?

[dhalbert]

Is this issue (boot keyboard vs standard usb composite device) why I can't wake a sleeping computer with neokey?

I do not think so. Instead, it's because we detect the USB shutdown when the sleep happens and then stop doing USB. I am not sure what we need to do to be able to wake. You can open a separate issue about that if you'd like, and we'll ask the USB expert on this.

[LostVector]

Wonderful ... thank you Dan. I will do that.

[dhalbert]

Using Win 10. It works well in the focussed app. BUT only sends(?) to foreground apps. I discovered this in OBS Studio, I can teach in the keystrokes my little Pico-Keyboard sends to OBS in the Hotkey section. And they work. The DO not work when I focus the App I want to record in OBS.

It turns out that OBS does special hotkey handling, to avoid needing focus. This is implemented differently on different operating systems. See this discord conversation (look backwards and forwards): https://discord.com/channels/327254708534116352/537365702651150357/901626930770612294

The issue that user had was partly using KeyboardLayoutUS.write() to send a ctrl-shift-somekey combination, instead of the raw keycode. @callimero, I am not sure whether that is a similar issue for you, but the keypress must be done in a certain way for Windows to detect it as a hotkey instead of a conventional keypress.

Closing because the problem does not lie with the library, I believe.

[callimero]

The issue that user had was partly using KeyboardLayoutUS.write() to send a ctrl-shift-somekey combination, instead of the raw keycode. @callimero, I am not sure whether that is a similar issue for you, but the keypress must be done in a certain way for Windows to detect it as a hotkey instead of a conventional keypress.

As you see in my code above I am not using KeyboardLayoutUS.write(). The Shortcuts work in OBS but only of it has focus. Other HID-Keyboard implementations (also a second off shelf USB-Keyboard) work on non focussed apps as well. I guess all so called "Stream Decks" work similar.

Reason for: Having OBS stream and present a different focussed app by Screen-Sharing.

Closing because the problem does not lie with the library, I believe.

This is sad. I don't think so.

[Neradoc]

@callimero when OBS is in the background, you need to hold the keys a little longer. When it's a human with a keyboard it often makes no difference, but if you type fast enough OBS will not see the background press. There are two solutions to that, add a sleep between press and release, or make the code hold the keys as long as the button is pressed (so your hand does the timing like a real keyboard). Try this:

while True:
    if taste.value:
        kbd.press(Keycode.SHIFT,Keycode.ALT,Keycode.LEFT_CONTROL,Keycode.K)
        # hold a while (adjust with tests)
        time.sleep(0.1)
        kbd.release_all()
        # debouncing (try using adafruit_debouncer or the CP7 keypad module for that)
        time.sleep(0.4)

PS: since adafruit_hid 5.1.0 you can use F13-F24, reducing interference with the front application shortcuts and the need for modifier keys.