search

keyboardio / Kaleidoscope

Firmware for Keyboardio keyboards and other keyboards with AVR or ARM MCUs.
http://keyboard.io
GNU General Public License v3.0
759 stars 259 forks source link

A way to interface software with Kaleidoscope ? #643

Open scarab5q opened 5 years ago

scarab5q commented 5 years ago

Hi there, I'm currently waiting for my model01 to arrive in the post and I was thinking about all of the ways I could customize it. I was just thinking it would be amazing if there was some kind of way that could allow external software to interact with Kaleidoscope and vice versa? Here are some examples of what I am talking about:

I've had a look around and I can't seem to find a plugin/API that could do this. It's entirely possible I'm missing something so if I am, please could someone point me in the right direction? If there isn't something like this I was imagining this could take the form of either some kind of API that could expose certain functionalities of kaleidoscope or some kind of daemon that could detect that a kaleidoscope based keyboard is detected and then allow certain software to interact with the keyboard through an API.

If this functionality doesn't exist or is being worked on, I would be very happy to get involved with building it or helping to build this.

obra commented 5 years ago

It's something that can kind of be done in a pretty hacky way today. We'd absolutely love the help. Most of the work will be on the computer side.

@algernon has written some commandline examples that use our Focus API (which is a serial protocol) to do -some- LED updates. What I'd -love- to see is a local REST API for LED updates. I -think- one right answer might be to implement the Das Keyboard LED REST API. https://www.hanselman.com/blog/LightingUpMyDasKeyboardWithBloodSugarChangesUsingMyBodysRESTAPI.aspx is a blog post about using their API to light up their keyboard. In an ideal world, we'd also implement Razer Chroma, though I don't believe that's publicly documented. ᐧ

On Thu, May 9, 2019 at 2:37 AM scarab5q notifications@github.com wrote:

Hi there, I'm currently waiting for my model01 to arrive in the post and I was thinking about all of the ways I could customize it. I was just thinking it would be amazing if there was some kind of way that could allow external software to interact with Kaleidoscope and vice versa? Here are some examples of what I am talking about:

  • your window manager could change kaleidoscope to a specific layer when you focus on a specific program
  • when vim changes to insert mode kaleidoscope could activate tap dance for certain keys
  • when you hit a prefix in emacs the keys that can be pressed next light up in a specific colour
  • pywal could set the colour scheme of your keyboard
  • set your keyboard to a specific configuration based on the OS that it is being used at the moment.

I've had a look around and I can't seem to find a plugin/API that could do this. It's entirely possible I'm missing something so if I am, please could someone point me in the right direction? If there isn't something like this I was imagining this could take the form of either some kind of API that could expose certain functionalities of kaleidoscope or some kind of daemon that could detect that a kaleidoscope based keyboard is detected and then allow certain software to interact with the keyboard through an API.

If this functionality doesn't exist or is being worked on, I would be very happy to get involved with building it or helping to build this.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/keyboardio/Kaleidoscope/issues/643, or mute the thread https://github.com/notifications/unsubscribe-auth/AAALC2CLHQJE6DHDTYD6X7LPUPWHDANCNFSM4HLY2IIA .

scarab5q commented 5 years ago

Sounds interesting! Currently doing a whole bunch of work for uni but after wards I will have time to have a go at either the local rest API though I think it would be best to have a bit of a discussion about the design of it. Though I think just making this for LEDs seems like a bit of a waste. I think using something like this to allow program specific keybindings and the like could be really cool

scarab5q commented 5 years ago

But anyway I'll return to this once I have finished uni and have played around with my model01 for a bit. It's supposed to arrive today so I'm quite excited!

noseglasses commented 5 years ago

Hey guys, with great interest i am following you discussion. Please allow me some questions as a non-web developer.

How does the REST communication work? Is it that there is some http server process running on the host that translates the http client requests (like that one from hanselmann's script) to a serial communication with the keyboard? Would that mean that one could e.g. use the DasKeyboard/Razer Chroma configuration software to configure the LEDs of a Kaleidoscope driven keyboard?

Could the communication between a Razer Chroma and the http server be intercepted to analyze their protocol? Would such a "reverse-engineering" be legal?

Do I understand this correctly, we would need to implement a host-based server process that implements the REST-API and e.g. translates it to Focus commands?

Or is it that the DasKeyboard itself processes the http requests?

algernon commented 5 years ago

For a local REST API, I think the idea is to have a small http server that accepts API requests, and translates them to the appropriate Focus stuff. It wouldn't be implemented on the keyboard, but in a tool running on the host. With the @chrysalis-api/focus libraries, this should be relatively simple to do in Node. From what I can tell from the blog post linked above, Das Keyboard does pretty much the same thing: the API is implemented by the "Q desktop app".

I suppose Razer does something similar. I seem to recall someone already reverse-engineered their SDK, but I don't have a link handy at the moment.

noseglasses commented 5 years ago

For the host-based control of the keyboard appart from the LED colors, I have some ideas. I could imagine adding a remote control service to focus that can nearly control everything.

Imagine we rely on standardized macros FOCUS_REMOTE_CONTROL and FOCUS_CONTROL_OBJECT. This REMOTE_CONTROL macro would be implemented in the sketch, e.g. as

#define FOCUS_REMOTE_CONTROL \
   FOCUS_CONTROL_OBJECT(a_global_plugin_object, aGetter, aSetter) \
   FOCUS_CONTROL_OBJECT(b_global_plugin_object, bGetter, bSetter) \
   ...

it could then be use to auto generate a function that focus calls when setter or getter requests arrive from the host. 

FOCUS_CONFIGURE_REMOTE_CONTROL(FOCUS_REMOTE_CONTROL)

The FOCUS_CONFIGURE_REMOTE_CONTROL would auto-generate a dedicated callback function and register it with focus. The registered class-members would be given a contiguous remote-control integer id.

Focus would then accept data getter and setter requests from the host.

By this means, we could remote control nearly everything that has a global instance and features setters and getters. An alternative macro to FOCUS_CONTROL could be implemented to directly read/write to global variables or public struct members.

Through the setter/getter signatures we would be able to generate type-safe send/receive code on focus' device side.

Does that make sense?

scarab5q commented 5 years ago

I mean honestly this is sort of exactly what I was imagining. I too am not a web developer, so this makes allot of sense to me, I'm glad to see this thread got a bunch of attention!

noseglasses commented 5 years ago

I mean honestly this is sort of exactly what I was imagining

Good do hear.

One part of this design that I am still pondering on is to find a way to generate a mapping from remote control ids to data types that can be used on the host side to validate the type of data to be send. Otherwise this setup would be too brittle.

The critical thing is that only the compiler knows about the exact datatypes that the firmware objects' setters and getters provide/accept. That's why some sort of compile or language parsing process with the device compiler would be necessary to get this right. We could generate a set of dummy variables in a dummy elf-library. Those could then be read by a script that parses the output of objdump. We could wrap all this inside some sort of script tool. But that's not really nice as it will be hard to get it portable. it would be cooler to integrate it somehow into Arduino's build system.

noseglasses commented 5 years ago

After thinking this through a bit further, I decided to have a crack at a focus-based remote control plugin for Kaleidoscope. I think i found a decent way to at least partially abuse the compiler/arduino-builder to generate files that can be used on the host side to validate I/O.

@scarab5q, I hope you don't mind me doing this. The local REST-API part will remain untouched.

scarab5q commented 5 years ago

Mate, do what ever you want ! I'm just happy to see it be a thing

noseglasses commented 5 years ago

@scarab5q, here's a first sketch of a remote control plugin.

https://github.com/CapeLeidokos/Kaleidoscope-RemoteControl

I'd be happy to receive feedback, issue reports, ideas for improvement, ...

scarab5q commented 5 years ago

That looks pretty great ! Don't have allot of time at this very moment to test it out but when I do I'll be sure to give you some feedback, ideas and issues!

noseglasses commented 5 years ago

@obra, i've written a plugin that does zone-based key coloring via remote control. This could possibly be used by a future REST API or something similar.

It currently does not allow for dynamic zone definition but on the other hand allows the zones to be defined as sparse matrices of zone IDs.

i currently use it to signal incoming Mail from Thunderbird that emits a simple focus-test call.

FWIW, maybe we could rename focus-test to something less sketchy as it is IMHO a proper tool?