Keyboard input: Some shift-levels not working

[eigengrau]

After upgrading from 10292b0 to b70421e, I’ve noticed that keyboard input from some shift levels stopped working. E. g., mode_switch + n yields a hyphen on my keyboard layout. After upgrading to b70421e, these keys no longer produce any output in rofi.

git bisect start b70421e 10292b0 lead me to e6ca8ac.

Pertinent event sequence below.

KeyPress event, serial 28, synthetic NO, window 0x3c00001,
    root 0x49b, subw 0x0, time 3353489, (622,475), root:(716,543),
    state 0x0, keycode 66 (keysym 0xff7e, Mode_switch), same_screen YES,
    XLookupString gives 0 bytes: 
    XmbLookupString gives 0 bytes: 
    XFilterEvent returns: False

KeyPress event, serial 28, synthetic NO, window 0x3c00001,
    root 0x49b, subw 0x0, time 3353648, (622,475), root:(716,543),
    state 0x22, keycode 46 (keysym 0x2d, minus), same_screen YES,
    XKeysymToKeycode returns keycode: 20
    XLookupString gives 1 bytes: (2d) "-"
    XmbLookupString gives 1 bytes: (2d) "-"
    XFilterEvent returns: False

KeyRelease event, serial 28, synthetic NO, window 0x3c00001,
    root 0x49b, subw 0x0, time 3353816, (622,475), root:(716,543),
    state 0x22, keycode 46 (keysym 0x2d, minus), same_screen YES,
    XKeysymToKeycode returns keycode: 20
    XLookupString gives 1 bytes: (2d) "-"
    XFilterEvent returns: False

KeyRelease event, serial 28, synthetic NO, window 0x3c00001,
    root 0x49b, subw 0x0, time 3353952, (622,475), root:(716,543),
    state 0x22, keycode 66 (keysym 0xfe02, ISO_Level2_Latch), same_screen YES,
    XKeysymToKeycode returns keycode: 51
    XLookupString gives 0 bytes: 
    XFilterEvent returns: False

[DaveDavenport]

I was afraid of that. What to mask off and and not is not fully clear to me from the X11 documentation. Ill take another stab at this.

Thanks for reporting.

[DaveDavenport]

Note to self, somewhere 'e (combined chars) also stopped working, do a bisect to find when.

[DaveDavenport]

Can you check if last commit fixes it?

[eigengrau]

Note to self, somewhere 'e (combined chars) also stopped working, do a bisect to find when.

Note to non-self: Did you mean dead keys? I tried going way back in the history to bisect but couldn’t find a commit where they worked for me.

Can you check if last commit fixes it?

I tried 5f7694f, but the behavior is unchanged (this includes dead accent keys and my secondary shift level).

Something I just noticed is that e6ca8ac did not only break combinations with Mode_Switch, but also the keyboard levels for ISO_Level3_Shift, ISO_First_Group and ISO_Last_Group. Since most of these are used to produce characters not super relevant in the context of rofi (greek or maths characters), they probably wouldn’t be missed. But maybe this says something about what exactly broke.

[DaveDavenport]

That is odd, both the combined keys to get é and right alt+5 to get € work for me (tm). The first one was also clear why it broke in the move to cairo, so if that worked before the move it should work now.

Tried looking up the right documentation on this, haven't found it yet.

[DaveDavenport]

I setup keyboard using:

setxkbmap -variant altgr-intl

Then using AltGr " e I get ë in rofi, with altgr-5 I get € and altgr-? ¿. What more do I need to check?

[eigengrau]

Try setxkbmap de adnw (not my exact layout, but I can produce analogous issues there, and it ships with Xorg).

To reproduce some of the shift-level issues:

• Press right alt + right shift + ISO key C03
• This should produce the character ∃ in most Xorg clients.
• This will not produce any output in rofi.

To reproduce combining character issues:

• Press ISO key E12
• Press ISO key C03
• This should produce the character è in most Xorg clients.
• This will produce the character e in rofi.

Another way of reproducing combining character issues:

• setxkbmap de
• Press ISO key E12
• Press ISO key D03
• This should produce the character è.
• This will produce the character sequence é in rofi and set the input mark before the ©.

[DaveDavenport]

Thanks, being able to reproduce helps. (Last one works fine for me.)

[eigengrau]

Oh, in case you need to get back to your regular layout after loading de adnw ;)

[DaveDavenport]

I put the right command in my clipboard before switching :tongue:

It isn't rofi 'eating' things anymore, that is indeed fixed. Xutf8LookupString does not seem to return a string when I enter those combi's. (except the last one, that works fine now)

(that must be one of the worst emotion icons... )

[DaveDavenport]

This is curious.. alt-shift-c03 results in a arrow down here from Xutf8LookupString Capslock-alt-C03 results me in ∃, this seems to match with the picture from above.

[DaveDavenport]

now that I am off the wild goose chase trying to reproduce something that wasn't going to work anyway, I think it is clear what is going on.. will fix it later...

[DaveDavenport]

Can you test the latest commit, hopefully it doesn't break anything. I currently do not accept key input on Ctrl,Alt,SuperL,SuperR. Now also work the arrow keys under ieau.

This needs a lot of testing!

[DaveDavenport]

Please test last update. You can now specify SuperR, SuperL, MetaR, MetaL, AltGr, HyperR,HyperL. If it does not exist in keymap, it will print an error.

[eigengrau]

I just tried with 7dccd80, and the various shift levels seem to work properly again. Awesome! :+1:

Binding Meta/Super/Hyper in daemon mode also worked as expected.

Dead key combinations still produce quirky output (both with a de layout and with de adnw). It seems that this is unrelated to the earlier keyboard input issues. Rather, it appears as if the glyph produced by the key is decoded as if it were latin-1. E. g., dead_grave + e produces the character è, but in rofi, this will show as è. I suspected this might be mojibake, and this seems to be the case:

$ echo è|iconv -f latin1 -t utf-8
è

I don’t know how latin-1 enters the picture, though, since it happens with LC_ALL=en_US.UTF-8. And it’s mysterious that this only happens for dead key combinations. E.g., greek characters, etc., don’t result in mojibake.

[DaveDavenport]

That is very weird, as dead key stuff been working fine here on several different machines (ubuntu, debian and void). Internally everything should be processed as utf-8. (Xutf8LookupString (I think) is used to obtain the entered character.)

It has worked fine in the past (Before I broke it going to cairo) and after the above fix. I will ask more people, see if they can reproduce? Anything 'special' about your setup.

Thanks for reporting the bug and all the feedback, it is in a better state now.

[DaveDavenport]

@carnager can you test?

[DaveDavenport]

there is a collision between grave as case sensitive select and possible the dead key. Need a better default keybinding for this. Suggestions?

(actually it still works, hitting twice switches case sensitivity,e still gets è.)

[eigengrau]

That is very weird, as dead key stuff been working fine here on several different machines (ubuntu, debian and void). Internally everything should be processed as utf-8. (Xutf8LookupString (I think) is used to obtain the entered character.)

Yea, it’s weird. I’ve now noticed that when I create a minimal ~/.XCompose in ISO-8859-1, composing characters will work within rofi (but of course this breaks all other applications, which expect the file to be UTF-8 due to my locale setting).

[eigengrau]

I noticed that rofi seemed to not pick up my locale properly: I added printf(XLocaleOfIM(xic)); to create_window, and this showed that Xlib assumed a C locale for some reason, which would explain why it read XCompose as latin-1. I was able to fix composing characters (for me, at least) by initializing the locale before the input context is created. I don’t know much C, so this might not be the best way to do it.

[DaveDavenport]

owh. f.f. sake I knew about the setlocal stuff, do it in other applications. I feel stupid..

[DaveDavenport]

merged patch.

[DaveDavenport]

I am unsure if we should set LC_ALL as that is normally left blank (it is a magic override all). Maybe LC_CTYPE is better..

LC_CTYPE Character classification

and

LC_COLLATE String collation

[DaveDavenport]

Done it git, please let me know if this breaks things for you.

update: Done a little test, it seems LC_ALL, even if unset in environment does automagically work and update the rest:

so:

$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE=C
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

Does result correctly in:

$ rofi -show window -modi window
LC_COLLATE  = C 
LC_CTYPE    = en_US.UTF-8 
LC_MESSAGES = en_US.UTF-8 
LC_MONETARY = en_US.UTF-8 
LC_NUMERIC  = en_US.UTF-8 
LC_TIME     = en_US.UTF-8 
LC_ALL      = LC_CTYPE=en_US.UTF-8;LC_NUMERIC=en_US.UTF-8;LC_TIME=en_US.UTF-8;LC_COLLATE=C;LC_MONETARY=en_US.UTF-8;LC_MESSAGES=en_US.UTF-8;LC_PAPER=en_US.UTF-8;LC_NAME=en_US.UTF-8;LC_ADDRESS=en_US.UTF-8;LC_TELEPHONE=en_US.UTF-8;LC_MEASUREMENT=en_US.UTF-8;LC_IDENTIFICATION=en_US.UTF-8

So LC_ALL is automatically filled in, and therefor works. Long live magic.

[DaveDavenport]

@eigengrau Let me know when this bug report can be closed.

[eigengrau]

Working just fine for me on 65dd080. Thanks a lot! :+1:

[DaveDavenport]

Thank you for all your patience and making Rofi better :smile:

[DaveDavenport]

hmm every keybinding broke on my work desktop pc now.. not fun.

[eigengrau]

The text editing bindings, or -key-run, etc.? I went through the list from the man page now and they all worked, expect for Ctrl-Shift-Tab and Ctrl-grave (on 611c2c4).

[DaveDavenport]

naah, it is a numlock issue. Probably easy fix.. happy enough that I catched this now.

will look at the shift-tab / ctrl-grave

[DaveDavenport]

Fixed.. i was adding mod masks to a global combined mask, but I should check if that modmask is not numlock first.

[github-actions[bot]]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.