Feature request input method suport

[YangtseSu]

Could not input cjk without input mothod

[dvdhrm]

Input-methods are fairly easy to implement, but honestly, I have no idea how they work for CJK. I don't speak any non-European languages so I need some help here:

• What input methods do you use on X11?
• How do you want these to work in kmscon?

It would be possible to implement a totally new infrastructure, but as I am in no position to evaluate these, it would speed up the process significantly if people describe what they expect. Reusing X11 input-methods would be preferred, though.

Cheers David

[penk]

fcitx and ibus are two popular input method frameworks, the former one has fbterm support so might be a good starting point. Please let me know if you need help in evaluating CJK input method.

Best, penk

[yuyichao]

1. You probably don't want to reuse Input method protocol under X, it's broken. (literally).
2. For input method support, it will be better to implement a input method plugin (probably similar to what fbterm does.)
3. For features that need to be supported, having a look at the new wayland input method protocol will actually be a good starting point. Basically, these are what an input method need,
   1. For event handling: grab keyboard and send back unicode string. (required)
   2. UI: display candidate word list. The input method need to display a list of possible composite result this should also be display next to the cursor. (required) (probably sth like a overlay display and cursor location api for input method plugin can do).
   3. UI: display status, (showing the input method in use etc... not sure where this should be located) (optional).
   4. It will also be really nice to also export some info about the running process pid/name etc so that the input method can be automatically activated/deactivated in certain condition etc. (optional).

[wengxt]

Well, support doesn't have to land in kmscon, using a screen-like wrapper and process the key there is ok to do some basic support.

[dvdhrm]

Ok, first of all, kmscon isn't actually the right place to implement input-methods. Instead, applications running in kmscon should support it. This is the same as line-editing: gettext is implemented in the client, not in the server (kmscon). (this is probably what @csslayer mentioned)

However, I understand that people might still want the input-method in kmscon (at least for pixel-based (not cell-based) rendering). But note, that this will never work in all situations. That is, applications don't expect the PTY host to draw stuff they don't want. Applications might discard every input or even disable the cursor and leave it somewhere else. But, if we do it correctly, applications at least don't break if we provide IM in the server. As a side-node, we need keyboard-shortcuts to enable/disable IM. But I don't think we need to extend the wire-protocol to allow applications to control it.

@yuyichao I did follow the Wayland discussion, but I lost focus the last month. Anyway, the problem with it is, that it highly depends on the wayland protocol. We could, of course, design a similar protocol, but I don't see me in a position to do that. But it helps me to see what is needed, indeed. Anyway, ibus (as it is X11 independent) seems like the best first approach to me. If that's done, we can always add other methods.

It is really easy to render overlays in kmscon, so I will give this a try. It may take some weeks, though. Any help is welcome.

Thanks for the input! David

[yuyichao]

On Wed, Feb 27, 2013 at 10:21 AM, David Herrmann notifications@github.comwrote:

Ok, first at all, kmscon isn't actually the right place to implement input-methods. Instead, applications running in kmscon should support it. This is the same as line-editing: gettext is implemented in the client, not in the server (kmscon). (this is probably what @csslayerhttps://github.com/csslayermentioned)

If you are talking about having input method support in all cmdline tool (e.g. vim/emacs/sh....), not really. It is really impractical. What @csslayer talking about is to have another program creating a embedded pty and provide input method support for all clients running in it.

However, I understand that people might still want the input-method in kmscon (at least for pixel-based (not cell-based) rendering). But note, that this will never work in all situations. That is, applications don't expect the PTY host to draw stuff they don't want. Applications might discard every input or even disable the cursor and leave it somewhere else.

Well this is not a problem, as long as the input window is always draw above anything else, input method and it's overlay window will work just as in any x terminal emulator.

But, if we do it correctly, applications at least don't break if we provide IM in the server. As a side-node, we need keyboard-shortcuts to enable/disable IM. But I don't think we need to extend the wire-protocol to allow applications to control it.

Input method can handle this (shortcut to activate/deactivate) correctly (as long as all key event are sent to it first). When it is not activated, it will just send back the original key event. There is no need to extend the wire-protocol, just provide the process info to the input method and let the input method handle that.

@yuyichao https://github.com/yuyichao I did follow the Wayland discussion, but I lost focus the last month. Anyway, the problem with it is, that it highly depends on the wayland protocol. We could, of course, design a similar protocol, but I don't see me in a position to do that.

What I mean is just the name/use of requests/events, which I summarized in the list after that.

But it helps me to see what is needed, indeed. Anyway, ibus (as it is X11 independent) seems like the best first approach to me. If that's done, we can always add other methods.

Fcitx is also X11 independent. And for an input method support in the server, a input method framework (IMF) independent protocol is always the best.

It is really easy to render overlays in kmscon, so I will give this a try. It may take some weeks, though. Any help is welcome.

Thanks for the input! David

— Reply to this email directly or view it on GitHubhttps://github.com/dvdhrm/kmscon/issues/74#issuecomment-14179088 .

[wengxt]

@dvdhrm you could try to provides a similar / same interface as fbterm. As for concrete input method framework support, it's kinds of our stuff ( @yuyichao and I are both fcitx developer ), you don't need to bother it too much right now, we could finish client easy if interface is there.

If you don't know the details of fbterm: fbterm spawn a process and set a environment variable for click to get the socket, here is something provided by fbterm: https://github.com/fcitx/fcitx-fbterm/blob/master/src/imapi.c

One flaw in fbterm input method design is, for most input method these days, we receives ALL keyboard input before application, which is not possible in fbterm. fbterm itself hardcode a trigger key to change a mode state maintain in fbterm, while we would prefer the state will be always maintained on input method side.

For emergency case, kmscon could provide key to kill the input method client if client have some problem, but hope it should send all key to input method in normal case. Since input method is not a CJK-only stuff, and can be used by all languages: http://fcitx-im.org/wiki/File:Fcitx-Keyboard.png

And another thing is fbterm can only send key event to input method by hardware keycode, so there is always some ugly code do the key code to keysym conversion, and some key info is not possible to get. It would be great if kmscon can support xkb style key translate, but that's not a big problem though.

[dvdhrm]

Thanks for the feedback! I'll just skip most replies as I mostly agree with you (and everything else is just bike-shedding).

So, I've just created an API proposal: https://gist.github.com/dvdhrm/5051547

The idea is, that every Input-Method (IM) engine should implement this API. In kmscon I will then use this API to get IM contexts and sessions. The function prefix is "uterm_*" as I think moving this into the kmscon-internal "uterm" library is the best way to get this done. You don't have to know the internals, though... Note that the IM engine runs in the same process as kmscon. I cannot see any reason to move this into a separate process, do you? (kmscon supports module-loading during runtime).

A uterm_im context is a global object which I allocate per kmscon-seat object (that is, all kmscon sessions in a single process on a single seat). There is normally only one uterm_im, but there might be more.

For each running terminal session, I create a new uterm_im_session object. I pass all input to this session, and the session notifies back (via a callback-function-pointer) what the terminal-session should do.

There is an event-loop implementation in kmscon (eloop) which you can make use of in the backend. However, if you want to write such a backend, feel free to create any pseudo-code and I will convert it to eloop.

What is missing? Well, basically a rendering-API. I need to know what kind of UI you want to draw. Is it really just the list of possible words/symbols? Or do you also draw on-screen keyboards? Or some tables or other stuff? I want to know what fcitx currently supports. I don't want any fancy ideas what we could do. If fcitx is extended in the future, then we can always extend the kmscon/uterm API. The current API just passes a 2D buffer (with alpha-channel) which is then shown by the terminal somewhere near the cursor position. But I can provide any other API that will do the text-rendering for you, so you don't need to do that in the backend.

If anything is unclear, just ask. Feel free to read the code, but asking here or on IRC is probably easier for you. I also don't want to force anybody of you to implement that. However, I really appreciate your feedback and would like to get comments what kind of API you would like to have if you were going to implement such a backend. Because this will then also make it easier for me to write the backends myself (if nobody else wants to do that).

Thanks a lot for the feedback! Considering that you two work on fcitx, I will definitely try to implement it first. It's a lot easier with direct contact to the developers. Thanks! David

[wengxt]

About GUI ability

As for on-screen keyboard.. doesn't kmscon support touch? if not, we don't need to bother that. I guess kmscon want o always focus on being a console instead of yet-another-display server with full featured toolkit.

Most input method framework, they use some "fixed" fields for displaying text, the engine inside framework just "fills up" those fields, and let the framework to choose the way to draw the real UI.

Since we are already in console, I guess we don't want to introduce that too much complexity, http://blip.tv/csslayer/fcitx-fbterm-demostration-5495807 Here is an old video I made for fcitx-fbterm, as you can see, it gives us the ability to get the position of the cursor, screen size, and be able to draw a background to show a simple window, and draw text enough.

The reason for those is we don't want to draw anything outside the screen, and handle the screen edge in a clever way to make sure it doesn't overlap with text we want to see, you might say the screen edge can be handled on kmscon side, which is true, I don't really have any preference on this decision though.

About Eventloop

As for event loop, it's a homemade event-loop for glib based? Either is ok for me, since I'm quite familiar with write dbus in custom event-loop, though fcitx also have a dbus client library which is based on glib (used by fcitx-fbterm). I guess that's a reason why fbterm use process separation because of toolkit they don't want to introduce into fbterm.

About ability to forward a key event

The other thing I want is, be able to send key event back to kmscon, this is important to make everything in an asynchronous flavor and support more feature.

The usual sequence is: display server -- (key-event) --> input method --- (absorb the event if required) ---> display server.

To make it asynchronous, the key-event maybe queued up and input method will always absorb the event at the beginning, and wait for server to say "put it back" or "absorb it". So there need to be an ability to put event back to kmscon later.

So actually, (*event) will always return true (or false depends on how you define the meaning) in this case. (here is an example for this process: https://github.com/fcitx/fcitx/blob/master/src/frontend/gtk2/fcitximcontext-gio.c#L523 the _use_sync_mode is false by default)

About commit the string

Here is something similar inside fcitx, some interface are really optional you may not want to care (for example, surrounding ones, preedit ones, things related to geometry), but definitely we need a way to commit text/string to kmscon, which seems is not in your interface right now. If we can have something similar with "forward key" as I just mentioned, that would be great. https://github.com/fcitx/fcitx/blob/master/src/lib/fcitx/frontend.h#L107

(As for IRC, I would prefer to use it after we are agree on a point after we don't need this kinds of long discussion :) , email and issue list are better for throughput and irc for latency IMHO )

[Aetf]

What is the current state of input method support? Any progress or is this feature postponed?

[dvdhrm]

Most current work was on systemd-console. I talked to the Debian accessibility people this week during XDC14 and all I need is a properly documented dbus API of your input-method. I'll make it work then.

[Aetf]

I'm not a developer of any input-method, I just use fcitx and want to continue to use it in kmscon ;)

After looked around in fcitx source code, I found this: https://github.com/fcitx/fcitx/blob/master/src/ui/kimpanel/kimpanel.c This is a DBus-based user interface for fcitx and at least supported methods and arguments can be found.

However according to the discussions made above I think maybe this is not enough though :disappointed:

[chengjianwen]

The post was very old, but I want to say that I finished a simple CJK input method based on kmscon. I upload the source to here

[nansun]

The post was very old, but I want to say that I finished a simple CJK input method based on kmscon. I upload the source to here

I compiled and tested it, it works. Thank you! The main feature works. The key configuration can then be polished to align with the most commonly used pinyin input method.

[nansun]

although the characters are not displayed at the right position, uim-fep did work under kmscon. perhaps we can fork some code from uim-fep. https://github.com/uim/uim/tree/master/fep

[chengjianwen]

I'll try to understand the uim, and seems it's a big one and have much features.

[htqx]

ibus not working in kmscon

[boslad]

@chengjianwen Your solution is very good. The problem of inputting individual Chinese characters has been solved, but how to solve the problem of inputting phrases? Maybe, we should modify the Kmscon_pinyin.h?