Suggestion: KVM switch support

lars18th commented 4 years ago

Maintainer edit from @mtlynch: Full list of compatible/incompatible KVMs is on the wiki.

Hi,

I use a commercial KVM-IPdevice connected to a regular (not cheap) KVM switch for controlling my home servers. So, I'm interested if this project can support the connection to one HDMI based KVM switch.

Anyone has tested it?

Futhermore, please can you add support for an ICON in the WEB UI to send a custom user key press? The reason is to map the "switch" keystroke to a button, as actually the "Scroll Lock" is used in several KVM switches. For example the idea is to send this with a button: < SCROLL > + delay 10ms + < SCROLL > + delay 10ms + < F1 >.

I hope you want to consider it. And congratulations for this great project! 👍 Regards.

somik123 commented 4 years ago

The post was posted just today. I've all the parts on order for my home server. Once they arrive from china in a week, I can let you know. I'm using a generic 4 port HDMI KVM switch though.

As it is emulating a USB keyboard and transmitting the HDMI, I dont see why it shouldn't work.

lars18th commented 4 years ago

Hi @somik123 ,

As it is emulating a USB keyboard and transmitting the HDMI, I dont see why it shouldn't work.

I see almost these reasons:

If the HDMI resolution reconnection requires to be done every time the switch selector changes. Perhaps this generates troubles with the HDMI capture device or with the software process.
If the KVM switch doesn't emulate the keyboard/mouse and instead it connects/disconnects the target devices to the connected server, then it's very possible that the keyboard emulation fails. I see a lot of cheap KVM switches that fails because this stupid working model... However, if the switch emulates internally the keyboard/mouse of each of the servers connected, then I don't see trouble reasons.

I hope it will work for you. Please, share with us your experience. Regards.

mtlynch commented 4 years ago

Thanks for reporting this @lars18th.

Futhermore, please can you add support for an ICON in the WEB UI to send a custom user key press? The reason is to map the "switch" keystroke to a button, as actually the "Scroll Lock" is used in several KVM switches. For example the idea is to send this with a button: < SCROLL > + delay 10ms + < SCROLL > + delay 10ms + < F1 >.

Can you tell me more about your use csae? Like you'd want to script a key sequence?

You can currently tell TinyPilot to send control characters on the next keypress even if you're not physically pressing them:

So for example, if you enable the buttons "Ctrl" and "Alt" in the web UI and then hit the "Del" key, TinyPilot will send Ctrl+Alt+Del to the target system.

somik123 commented 4 years ago

Can you tell me more about your use csae? Like you'd want to script a key sequence?

For HDMI KVM switches, you have 4 PCs connected to a HDMI KVM via both HDMI and USB.

You connect 1 monitor, 1 keyboard and 1 mouse to the KVM.

Normal HDMI KVM switches (not HDMI splitters or HDMI mergers) uses a specific key combination, such as [ScrollLock] [NumLock] to swich the input to a different PC. For example, if you are using PC1 and want to switch to PC2, you use [NumL] [NumL] (or [ScL] [ScL] [F1] in his model) and the KVM will swich to the next PC or cycle through them.

When that happens, the keyboard + mouse + monitor is temporary disconnected (for up to 2s for cheapo models) and gets connected to the next PC's USB and HDMI inputs. Using a teensy keyboard emulation, I did manage to get it working fine, so I think keyboard emulation will work. As for change in resulation between the PCs, I got no idea if that will work...

mtlynch commented 4 years ago

Thanks for the explanation, @somik123!

What can I add to TinyPilot to better support this scenario? Are there key sequences that currently aren't possible to send through TinyPilot?

somik123 commented 4 years ago

@mtlynch I was checking the JS keyboard event monitor implementation here: https://javascript.info/keyboard-events

Their JS code is:

kinput.onkeydown = kinput.onkeyup = kinput.onkeypress = handle;

let lastTime = Date.now();

function handle(e) {
  if (form.elements[e.type + 'Ignore'].checked) return;

  let text = e.type +
    ' key=' + e.key +
    ' code=' + e.code +
    (e.shiftKey ? ' shiftKey' : '') +
    (e.ctrlKey ? ' ctrlKey' : '') +
    (e.altKey ? ' altKey' : '') +
    (e.metaKey ? ' metaKey' : '') +
    (e.repeat ? ' (repeat)' : '') +
    "\n";

  if (area.value && Date.now() - lastTime > 250) {
    area.value += new Array(81).join('-') + '\n';
  }
  lastTime = Date.now();

  area.value += text;

  if (form.elements[e.type + 'Stop'].checked) {
    e.preventDefault();
  }
}

Only [Win] key is not processed by the webstite that is using the this. As well as Ctrl+Alt+Del and other hotkeys captured by third party applications. As such, i think you should not need to worry about keyboard capture events for now. I wont be able to test it until my HDMI capture device arrives thogh.

lars18th commented 4 years ago

Hi @mtlynch ,

Can you tell me more about your use csae? Like you'd want to script a key sequence?

I feel that @somik123 will provide more valuable feedback. In any case, my simple suggestion is to provide some BUTTONS in the UI to send the keys corresponding to the change of the Switch. If you want you can create specific buttons, or if you prefer a simple User Configurable buttons.

Regards.

somik123 commented 4 years ago

@lars18th, seems to work fine with my KVM switch. The keyboard does cut out from time to time after a pc-switch-over, but not sure if it is cause of my cheap KVM or the tinypilot.

Lack of mouse support sucks though as most keyboard-only systems also support remote ssh...

mtlynch commented 4 years ago

@somik123 could you share the name of the KVM switch you're using? You'd be the first to confirm compatibility with a KVM switch, so I'll document in the README that there's a switch that's tested compatible with TinyPilot and link others to it.

somik123 commented 4 years ago

This is the HDMI switch I am using. It switches input by pressing [ScrollLock] or [NumLock] twice within 1000ms (1s).

FJGEAR-4-Port-USB-HDMI-KVM-Switch

Here is a proof that both Numlock and Scroll Lock passthrough works. I'm using my PC to connect to tinypilot, which is connected to my windows laptop. The laptop browser is showing keypresses.

Screenshot at 2020-08-01 21-13-54

lars18th commented 4 years ago

Hi,

Here is a proof that both Numlock and Scroll Lock passthrough works.

Great! However, it will be interesting to have one button to send the KEY combination with just one click. :wink:

The keyboard does cut out from time to time after a pc-switch-over, but not sure if it is cause of my cheap KVM or the tinypilot.

If the KVM doesn't emulate the keyboard, then is normal the glitch. Please, check the LOG of the TinyPilot to see if when you switch the USB keyboard is initialized. In this case, to solve the glitch will be necessary to add some code.

Regards.

somik123 commented 4 years ago

Great! However, it will be interesting to have one button to send the KEY combination with just one click.

But since there are so many combinations for different types of KVMs, it'll be difficult to hard-code one for each. Easier to give a "setup" option for the user to configure own hotkeys.

If the KVM doesn't emulate the keyboard, then is normal the glitch. Please, check the LOG of the TinyPilot to see if when you switch the USB keyboard is initialized. In this case, to solve the glitch will be necessary to add some code.

I dont think this KVM emulates the keyboard... I'll check the logs next time this happens, but how do i bring up the tinypilot logs from pi shell?

mtlynch commented 4 years ago

Logs are available via

sudo journalctl -u tinypilot

madmedicine commented 4 years ago

First I want to say "Awesome project!"

I was curious to see if it worked with my KVM, a Trendnet TK-803R. Installed Tinypilot last night and works out of the box with my KVM!

Switching inputs are done with "scrolllock,scrollock, number key" and can switch to any inputs 1 to 8 easily.

ubenmackin commented 4 years ago

But since there are so many combinations for different types of KVMs, it'll be difficult to hard-code one for each. Easier to give a "setup" option for the user to configure own hotkeys.

This would be great! If some how we could define a text file (or some configure file) that had a key sequence to be applied via the interface as a button.

I think not only for these kind of KVMs, but if I could define some common text inputs (like username [Enter] password [Enter]), a button press to execute would be a nice feature.

ubenmackin commented 3 years ago

I recently got one of these FJGEAR kvm switch. When I try and do the "scrolllock scrollock" twice to switch the switch, the KVM shows that it has switched (the LED light updates), but tiny pilot throws some errors:

Dec 26 18:23:24 pikvm python[572]: [2020-12-26 18:23:24,263] ERROR in socket_api: Failed to forward mouse event: Failed to write to HID interface: /dev/hidg1. Is USB cable connected?
Dec 26 18:23:27 pikvm python[572]: [2020-12-26 18:23:27,105] ERROR in socket_api: Failed to forward mouse event: Failed to write to HID interface: /dev/hidg1. Is USB cable connected?
Dec 26 18:23:30 pikvm python[572]: Process Process-7:
Dec 26 18:23:30 pikvm python[572]: Traceback (most recent call last):
Dec 26 18:23:30 pikvm python[572]:   File "/usr/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
Dec 26 18:23:30 pikvm python[572]:     self.run()
Dec 26 18:23:30 pikvm python[572]:   File "/usr/lib/python3.7/multiprocessing/process.py", line 99, in run
Dec 26 18:23:30 pikvm python[572]:     self._target(*self._args, **self._kwargs)
Dec 26 18:23:30 pikvm python[572]:   File "/opt/tinypilot/app/hid/write.py", line 18, in _write_to_hid_interface_immediately
Dec 26 18:23:30 pikvm python[572]:     hid_handle.write(bytearray(buffer))
Dec 26 18:23:30 pikvm python[572]: BrokenPipeError: [Errno 108] Cannot send after transport endpoint shutdown
Dec 26 18:23:46 pikvm python[572]: [2020-12-26 18:23:46,535] ERROR in socket_api: Failed to forward mouse event: Failed to write to HID interface: /dev/hidg1. Is USB cable connected?
Dec 26 18:23:50 pikvm python[572]: [2020-12-26 18:23:50,249] ERROR in socket_api: Failed to forward mouse event: Failed to write to HID interface: /dev/hidg1. Is USB cable connected?

@somik123 I know it has been 6 months, but are you successfully still using this KVM?

WarheadsSE commented 3 years ago

I've opened #507 as a related item and am working through other modifications I have had to make.

Parts listing: https://gist.github.com/WarheadsSE/729633f0ecc6e5453f14485f2d09c87f

mtlynch commented 3 years ago

@WarheadsSE - Did you have to change the HID initialization part? I bought the same KVM, and it only works if I change the init-usb-gadget script to this version that's keyboard only and I plug into the keyboard input port.

I was never able to get keyboard, mouse, and hotkey switching working all at once.

Here were the results of my tests:

CKLau 4-Port HDMI KVM

Device	KVM port	TinyPilot Version	Forwards keyboard input?	Forwards mouse input?	Hotkey switching works?
TinyPilot on Pi 4B	USB 2.0 hub	Standard	Yes	Yes	No
TinyPilot on Pi 4B	USB 2.0 hub	Modified to be keyboard-only	Yes	No	No
TinyPilot on Pi 4B	Keyboard	Standard	No	No	No
TinyPilot on Pi 4B	Keyboard	Modified to be keyboard-only	Yes	No	Yes
Real USB keyboard	USB 2.0 hub	N/A	Yes	No	No
Real USB keyboard	Keyboard	N/A	Yes	No	Yes

WarheadsSE commented 3 years ago

Did you have to modify the gadget init ?

Yes, I had to disable the mouse’ descriptor. I’m working on sorting this out, because using the current init, I can make either one work on the keyboard input, but not together. I have a usb on/mouse combo dongle that works fine, which leads me to believe there is a slight glitch in the descriptors when in multi device behavior, probably due to minimal or overly simple but strict compliance to the USB specification.

I intend to sort this out and submit a patch for this as well, to the ansible repository, now that it lives over there. -- Jason Plum WarheadsSE

mtlynch commented 3 years ago

I've added a page to the wiki to track compatible/incompatible KVMs:

https://github.com/mtlynch/tinypilot/wiki/KVM-compatibility

glendady commented 3 years ago

I have an Avocent IP-KVM 16 port switch (bought used on eBay), which I cannot use the KVM over IP functionality because of a certificate issue, and I cannot upgrade the cert. So until buying the TinyPilot I used it as a physical KVM where I need to be at the console. I was hoping that I could use the TinyPilot to connect to the Avocent switch and be able to use it over the wire. What I have noticed is to bring up the OSD I have to use the virtual keyboard supplied by the TinyPilot device. Pressing the Scroll Lock key on my keyboard does not active the OSD. Also, the Mouse does not make it to the connected computer. Any ideas how I could at least get the mouse functionality to work over the Avocent 16 port switch?

mtlynch commented 3 years ago

@glendady - It's unfortunately unlikely to work with that KVM switch. I added more detail in #697.

taylor-schneider commented 2 years ago

I wanted to add my two cents on the issue:

I would like to see the UI augmented in two ways:

On the page where we see the screen for the connected machine, I would like the View dropdown menu to have an item "Show KVM". If selected we would see a rectangular array of numbered buttons which can be clicked to trigger the kvm to switch to the desired host.
A new item "KVM Configuration" is added to the System drop down. This configuration takes us to a new page. On this page we configure the KVM connection. It should have a list which can be modified. Each row corresponds to a switch port on the KVM. The user can add or remove as necessary. For each row the user has the option of specifying the path to a script which can be executed when the button on the KVM panel is clicked (mentioned in point 1).

For example, i have a testmart.sh script (thanks to pschmitt) which allows me to programatically switch between ports using this utility.

With the solution mentioned above it would be seamless, rather than me needing to ssh into the tinypilot to run the ssh script I have installed. It would also be extendable. Users could ad config files / scripts to add support for various popular switches.

tiny-pilot / tinypilot

Suggestion: KVM switch support #78

CKLau 4-Port HDMI KVM