search

keyboardio / Chrysalis

Graphical configurator for Kaleidoscope-powered keyboards
https://github.com/keyboardio/Chrysalis#chrysalis
GNU General Public License v3.0
496 stars 65 forks source link

Keyboard-Simulation with Chrysalis #397

Open noseglasses opened 5 years ago

noseglasses commented 5 years ago

Keyboard-Simulation with Chrysalis

This is not an actual feature request. Instead I want to ask for the community's opinion about an idea that I recently had, which is adding keyboard simulation to Chrysalis.

Keyboard Simulation

First let me explain what I mean when referring to keyboard simulation.

The following scenario is what I have in mind: How about giving the user the ability to try out (here I mean to play with the actual configured keyboard) any configuration (keymaps, plugins, LED modes, ...) that Chrysalis supports without the requirement to flash to the actual keyboard - in fact even without the need to have the Kaleidoscope-keyboard that is under configuration in Chrysalis?

After selecting plugins, configuring the keymap and possibly also the LED colors, the user would push a button "simulate" that would make Chrysalis switch to a mode of operation where it simulates the entire keyboard in realtime. Every key in Chrysalis would display the Key(code) that is currently associated with a key and the key LEDs would behave exactly as they would with the real keyboard. Any LED modes would be rendered in realtime.

A Prototype

This may sound like science fiction but it isn't as this short video demonstrates. Please excuse the bad quality ;-) I shot this video some days ago to demonstrate what can be done with kaleidoscope's virtual core. That is a rarely used feature that fell into a state of disrepair while development of the firmware rapidly (too rapidly) progressed in the last year.

As firmware simulation is a highly valuable feature not only for realtime simulation but especially for integration testing, development and debugging I am currently working on getting it production ready again.

The prototype application shown in the video does the following. Via a dedicated plugin the physical keyboard sends the state of its keys to the host (serial communication) at the end of each cycle. The fancy part happens on the host. There the firmware simulator running in a loop reads the keyboard's keystates from stdin at the beginning of each cycle and considers them as changes of the virtual keystates - just like a virtual user hitting virtual keys. Then it calls Arduino's loop function and react on key action and change LED colors that are depicted in the console.

I am currently working on a simulation/testing API that plays a key role.

Integrated Keyboard Simulation in Chrysalis

Now back to Chrysalis and my ideas of linking it with the simulator...

I am impressed by those very nice graphical visualizations of the keyboard in Chrysalis' GUI. It would be a perfect replacement for my primitive 80s style console-visualization. And that's what brought me to the idea that it might not only be a nice toy but add real value to Chrysalis to provide realtime simulation capability.

To let the user play with its configured keyboard before or without actually flashing it to the keyboard would IMHO be really cool.

What would be needed from Chrysalis side? I do not have an overview about what is already there, so I am going to list those points that I can see possibly missing and hope that some of the Chrysalis devs might answer what is there and what isn't.

Chrysalis would need to

It might be cool to allow the user to use any type of physical keyboard to control the simulator, not necessarily the simulated one. That way a user could try out a keyboard before actually buying it.

What do you think? Any opinions are welcome.

Here you can follow the progress of the ongoing work on the simulator.

algernon commented 5 years ago

Chrysalis would need to

  • generate a sketch file based on the users current configuration in the GUI

This is something we considered before, and something we want to see at some point.

  • compile this firmware sketch by running kaleidoscope builder

This is also something we considered, but... it has a number of complicating factors: we want Chrysalis to be stand-alone, and ship with ~everything required. For compilation, we'd need to ship with Arduino and Kaleidoscope and whatnot. It would also tie Chrysalis to a particular Kaleidoscope + Arduino combo. Distribution would also be fairly problematic (license-wise too, as we're GPL-3-only, while Arduino has some unclear parts).

Instead, we were thinking about running a compilation service with an API, where Chrysalis could send the sketch, and would get a compiled hex back (the server side of that would be open source as well, of course). This way we avoid a lot of the problems with bundling Kaleidoscope and/or Arduino with Chrysalis, and we also save the end-user from having to install either of those.

  • open a .dll/.so (the simulator dll generated by kaleidoscope builder) (am I right that to open dlls is possible with electron?)

I think this should be possible. That makes the rest of your points a possibility as well.

It might be cool to allow the user to use any type of physical keyboard to control the simulator, not necessarily the simulated one. That way a user could try out a keyboard before actually buying it.

A precursor to this simulator could be support for virtual hardware, I think. Doing that would require Chrysalis to deal with hardware that isn't necessarily Serial and Focus based.

I do like the simulator idea, and the video you linked is impressive as it is, in all its ASCII glory.

noseglasses commented 5 years ago

Instead, we were thinking about running a compilation service with an API, where Chrysalis could send the sketch, and would get a compiled hex back (the server side of that would be open source as well, of course). This way we avoid a lot of the problems with bundling Kaleidoscope and/or Arduino with Chrysalis, and we also save the end-user from having to install either of those.

That sounds cool! For the simulator it does not matter, anyway where the sketch is build.

A precursor to this simulator could be support for virtual hardware, I think.

I suspect that what you refer to as virtual hardware is almost the same as what I tend to call simulator. As I am at home in simulation business I might overuse the term simulator here and there.

The simulator library i am currently working on is a wrapper for the virtual hardware. it takes care of faking input and intercepting output and also adds some integration testing infrastructure.

Doing that would require Chrysalis to deal with hardware that isn't necessarily Serial and Focus based.

There are pseudo-teletypes on linux. With their help it could become possible for the simulator to act as a serial device. No idea about Windows, though.

But that part is actually what I was thinking off to solve by having Crysalis load a dll that contains the simulator. It could then just use the simulator's API instead of exchanging focus commands. Registering a callback for LED color changes and calls to the simulator's pressKey(...) and releaseKey(...) methods would be all it needs.

Anyway, that's all science fiction and i am sure that there's a lot more important features of Chrysalis that deserve to be finished before a simulator would be worth being added. My intention was just to inform the Chrysalis community about what is going on in the virtual hardware/simulator business. it would definitely be cool to bring both worlds together one day.

I do like the simulator idea, and the video you linked is impressive as it is, in all its ASCII glory.

Thanks. :smirk: