Open pete-otaqui opened 3 years ago
Sorry for missing this! I typed up a response a few days after you created the issue, but I must have accidentally closed the tab without publishing it.
It's a bit of a hectic week, but I should be able to respond to this next week.
Sorry again for the delay on this @pete-otaqui!
I actually get keystroke events for the media buttons on my keyboard in any window on Chrome and Firefox on Win10:
document.addEventListener("keydown", (evt) => {console.log(evt);});
VM120:1 KeyboardEvent {isTrusted: true, key: "AudioVolumeMute", code: "", location: 0, ctrlKey: false, …}
VM120:1 KeyboardEvent {isTrusted: true, key: "AudioVolumeDown", code: "", location: 0, ctrlKey: false, …}
VM120:1 KeyboardEvent {isTrusted: true, key: "AudioVolumeUp", code: "", location: 0, ctrlKey: false, …}
VM120:1 KeyboardEvent {isTrusted: true, key: "MediaPlayPause", code: "", location: 0, ctrlKey: false, …}
VM120:1 KeyboardEvent {isTrusted: true, key: "MediaPlayPause", code: "", location: 0, ctrlKey: false, …}
keydown { target: body.illustrated, key: "MediaPlayPause", charCode: 0, keyCode: 0 }
keydown { target: body.illustrated, key: "AudioVolumeDown", charCode: 0, keyCode: 182 }
keydown { target: body.illustrated, key: "AudioVolumeUp", charCode: 0, keyCode: 183 }
keydown { target: body.illustrated, key: "AudioVolumeMute", charCode: 0, keyCode: 181 }
keydown { target: body.illustrated, key: "AudioVolumeMute", charCode: 0, keyCode: 181 }
Is behavior different on other OSes? This might be as simple as adding mappings to js_to_hid.py.
Hi, apologies for the delay in replying.
Yes, the behaviour is different in macOS - media key presses are not captured at all by document.addEventListener
.
Perhaps a good solution is to provide media keys in the virtual keyboard?
Media keys that exist on some keyboards, for example "Play/Pause", "Next Track" and "Previous Track could be handled by TinyPilot.
In my case, I am actually using TinyPilot to remote control a variety of media devices. These are usually managed with a bluetooth / IR remote control, or other more-exotic-than-a-keyboard control device, but generally do have support for a keyboard being plugged in. There are a couple of types that only provide full functionality via media keys when using a physical keyboard.
HID Codes
I haven't verified these, but I found a reference to the HID codes with this information:
I notice that you already have the Play/Pause HID code in
keycodes.py
, however I'm not sure how it would be used since the virtual keyboard does not have this key, and I don't believe the UI will capture it and forward it on - please do let me know if I'm wrong about that.Physical and Virtual Keyboard
Capturing these key strokes from a physical keyboard in a browser is different to normal keypresses. Instead of
addEventListener
, one must use theMediaSessionApi
.In any case, it seems that the couple of browsers I tested (Firefox and Chrome on macOS) ignore these action handlers if there is no "active" media session (see note*)
That being said, it would obviously be possible to add media keys to the virtual keyboard.
Updated socket API
The "real" callback arguments for the action handlers in the
MediaSessionApi
are not like normal key events, but much more simplistic:Between the different event types, and the different callback arguments, it seems as though it would be wrong to extend the existing
keystroke
socket API with "fake" media key data. An alternative option would be to introduce a new part to the socket api, specifically for this purpose.Ultimately pressing these keys could lead to emitting something like the following:
There would then presumably need to be an equivalent
request_parsers/media_keystroke.py
- this file could be simpler than the existingkeystroke.py
parser, since it would not have to handle any modifiers. I guess thatjs_to_hid.py
would also need aconvert_media(mediaKeystroke)
similar to the existingconvert(keystroke)
. And some other updates, most notably tosocket_api.py
to implement the rest of the parts.--
*note If I load a youtube video in the browser, the "Play/Pause" button will work to control the video. If I then use the developer console to set my own action handler, it takes over and my handler is called instead of controlling the video when I press the button. However - if I add the same handler on an arbitrary non-video-containing page it does nothing at all.