TurboVNC / turbovnc

Main TurboVNC repository
https://TurboVNC.org
GNU General Public License v2.0
789 stars 142 forks source link

Keyboard mapping issue #106

Closed amitkumars1985 closed 7 years ago

amitkumars1985 commented 7 years ago

VNC : TurboVNC 2.1.1 Server OS : RHEL 6.7 Client OS : Windows 10

It seems that VNC Client doesn't recognize some of the key press events when done using different keyboard format. Here are the scenarios :

  1. In Finnish Keyboard pressing Ctrl+- ( Dash ) also triggers Alt key press event :

In Finnish Keyboard :

KeyPress event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688475879, (412,212), root:(1171,599), state 0x10, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyPress event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688476327, (412,212), root:(1171,599), state 0x14, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyPress event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688476327, (412,212), root:(1171,599), state 0x15, keycode 20 (keysym 0x5f, underscore), same_screen YES, XLookupString gives 1 bytes: (1f) "" XmbLookupString gives 1 bytes: (1f) "" XFilterEvent returns: False

KeyRelease event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688476327, (412,212), root:(1171,599), state 0x15, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

KeyRelease event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688476406, (412,212), root:(1171,599), state 0x14, keycode 20 (keysym 0x2d, minus), same_screen YES, XLookupString gives 1 bytes: (2d) "-" XFilterEvent returns: False

KeyRelease event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688476710, (412,212), root:(1171,599), state 0x14, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

In Standard English Keyboard :

KeyPress event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 689157909, (165,229), root:(751,479), state 0x0, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyPress event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 689158281, (165,229), root:(751,479), state 0x4, keycode 20 (keysym 0x2d, minus), same_screen YES, XLookupString gives 1 bytes: (2d) "-" XmbLookupString gives 1 bytes: (2d) "-" XFilterEvent returns: False

KeyRelease event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 689158373, (165,229), root:(751,479), state 0x4, keycode 20 (keysym 0x2d, minus), same_screen YES, XLookupString gives 1 bytes: (2d) "-" XFilterEvent returns: False

KeyRelease event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 689158456, (165,229), root:(751,479), state 0x4, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

  1. In Finish/Dannis keyboard AltGr key produces extra key press events

In Finnish Keyboard :

KeyPress event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688477919, (412,212), root:(1171,599), state 0x10, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyPress event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688477920, (412,212), root:(1171,599), state 0x14, keycode 113 (keysym 0xffea, Alt_R), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyRelease event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688478039, (412,212), root:(1171,599), state 0x1c, keycode 37 (keysym 0xffe3, Control_L), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

KeyRelease event, serial 35, synthetic NO, window 0x4000001, root 0x69, subw 0x0, time 688478039, (412,212), root:(1171,599), state 0x18, keycode 113 (keysym 0xffea, Alt_R), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

In Standard English Keyboard :

KeyPress event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 687724724, (-68,62), root:(526,320), state 0x0, keycode 113 (keysym 0xffea, Alt_R), same_screen YES, XLookupString gives 0 bytes: XmbLookupString gives 0 bytes: XFilterEvent returns: False

KeyRelease event, serial 31, synthetic NO, window 0x2200001, root 0x69, subw 0x0, time 687724805, (-68,62), root:(526,320), state 0x8, keycode 113 (keysym 0xffea, Alt_R), same_screen YES, XLookupString gives 0 bytes: XFilterEvent returns: False

Is there any way to fix it and make it compatible with different keyboard formats.

dcommander commented 7 years ago

TurboVNC has traditionally been tested a lot on U.S., Swedish, and Australian keyboards but not as much on others, so there may be minor issues like this. I'll investigate.

dcommander commented 7 years ago

There might be a misunderstanding regarding what TurboVNC is supposed to do here. I should start by explaining the difference between keycodes and keysyms. A keycode is a numeric code corresponding to a physical key on the keyboard. A keysym is the symbol generated by a particular key sequence. The keyboard layout determines how the operating system translates keycodes into keysyms.

Unfortunately the RFB protocol cannot send keycodes without using an extension that mainstream VNC viewers and servers don't currently support (although I just created a feature request for it-- #108.) Currently, VNC viewers have to send keysyms to the server, and those keysyms depend on the client's keyboard layout.

In Finish/Dannis keyboard AltGr key produces extra key press events

That's because Windows translates the AltGr key into LCtrl + RAlt key events. This is true for all Windows applications, not just the TurboVNC Viewer. It is true that X11 translates AltGr differently (as a single ISO Level 3 Shift event.) However, there is no sane way for the TurboVNC Viewer to perform this translation, because it has no way of knowing whether the LCtrl + RAlt sequence was generated in response to AltGr on an international keyboard or whether the user literally pressed LCtrl + RAlt on a U.S. keyboard. Windows does not distinguish between those two key sequences, so there is no way for us to do so without hooking into the keyboard at a lower level and transmitting only raw keycodes.

In Finnish Keyboard pressing Ctrl+- ( Dash ) also triggers Alt key press event :

No, it triggers a Shift key press event, as your xev log clearly shows, not an Alt key press event. The reason is that, once again, VNC viewers transmit key symbols rather than key codes. Let's take the example of the "+" key symbol. On a Finnish keyboard, that key symbol can be generated by pressing the equivalent of the "-" key on U.S. keyboards, without pressing Shift. So if you press that key while the client is using a Finnish keyboard layout, the TurboVNC Viewer will transmit a "+" keysym to the server. The server must figure out how to generate a series of fake key events that will produce that symbol. If the server is using a U.S. keyboard layout, then it has to generate a Shift key event in order to produce a "+". This ordinarily will not matter. Unless the application you are running on the server is trying to hook into the keyboard at the low level and process keycodes itself (games, for instance, sometimes do this), then the application will respond the same regardless of whether the "+" symbol was accompanied by a Shift key event (U.S. layout) or not (Finnish layout.) If you're really worried about it, though, you can simply change the keyboard layout on the server to Finnish. In that case, the server will detect that it no longer needs to generate a Shift key event to produce a "+" symbol, and it won't do so.

To summarize, if you press a key sequence that generates a particular symbol (such as "£" or "+" or "é") on the client machine, you should expect that the same symbol will be generated on the server. However, the actual sequence of X11 events used to generate that symbol will vary depending on the client's and server's keyboard layouts. Also, the modifier keys (Alt, Ctrl, AltGr, Shift, etc.) are generally passed to the server as-is, which is why AltGr generates a LCtrl+RAlt key sequence on the server when using a Windows client. But the actual keysyms generated using AltGr should match between client and server. So if you press AltGr+3 with a Finnish keyboard layout on the client, you should get a British pound sign on the server, but the exact sequence of X11 events used to generate that British pound sign will vary depending on the server's keyboard layout, and you'll get a Ctrl and Alt X11 event accompanying the symbol if you're connecting from a Windows client. Again, that's a limitation of the RFB protocol that is going to exist in all VNC servers. There's a way around it, by using the QEMU extension, but it would take some time (and money) to develop and test that implementation.

amitkumars1985 commented 7 years ago

First of all I apologies for the mistake. it's Shift key not Alt Key as mentioned in below statement :

"In Finnish Keyboard pressing Ctrl+- ( Dash ) also triggers Alt key press event :"

The certain key combination I mentioned were actually required by users to work. Since it was not working with VNC client hence they started complaining and asking for solution. Also it seems to be problem only with Java Client not with Native Client. In Native client, we didn't observe any key mapping issue but we can't use it as it doesn't support encryption.

dcommander commented 7 years ago

OK, but please explain what that key sequence is supposed to do. In my testing, it didn't do anything.

amitkumars1985 commented 7 years ago

Here are 2 scenarios which we have testes :

  1. On Finnish Keyboard

In emacs Editor Ctrl+_ ( Underscore ) is used to undo which is achieved by pressing Ctrl+Shift+-(Dash) combination. In TurboVNC, since Ctrl button is triggering addition KeyPress event of Shift hence it is combination is achieved by pressing Ctrl+-(Dash ) ( No Shift Key Press ) while pressing Ctrl+Shift+-(Dash) combination doesn't work.

  1. On Danish Keyboard :

In Danish keyboard "|" ( pipe ) sign is achieved by pressing AltGr + "+=" key while in TurboVNC viewer AltGr + "+=" doesn't print "|" sign

dcommander commented 7 years ago

I can now reproduce the problem. Thanks for the additional information. The core of the issue is Java's fault. It unfortunately isn't doing the right thing when certain symbol keys are pressed with the Control-Shift modifier combination. For instance, when I press Shift-Minus on the Finnish keyboard, Java correctly returns the underscore character when I call the getKeyChar() method on the KeyEvent. However, when I press Control-Shift-Minus, Java returns 65535 (undefined) in the getKeyChar() method. I also tried using the AWTKeyStroke class, which supposedly has the ability to return a key character for any arbitrary key sequence (the idea being that, if a Control-Shift combination returns undefined, I could query the key character for the same combination without Control), but it doesn't even pretend to work properly. The best I'll be able to do is probably to assume that a dash/minus accompanied by a Control-Shift should always be interpreted as an underscore. That isn't universally true, but it seems to be the case on most keyboard layouts.

dcommander commented 7 years ago

Also, in the future, please describe the symptoms of the problem-- that is, how the issue is affecting end users and applications-- rather than leading with your own diagnosis. I always have to be able to reproduce a problem before it can be fully diagnosed and fixed, and I generally can't reproduce it without information about how the problem is affecting end users.

dcommander commented 7 years ago

Fix pushed. Please test ASAP, as the 2.1.2 release is imminent.

amitkumars1985 commented 7 years ago

Do I need to install 2.1.2 server as well or can just test it using java client?

dcommander commented 7 years ago

This particular fix was entirely in the Java viewer, but I strongly recommend updating the server as well. There were numerous bug fixes since 2.1.1. A couple of those bug fixes might have affected the other issues you submitted.

dcommander commented 7 years ago

More specifically, I need you to retest #102 with the latest & greatest server build, since there are several things in that build that might have worked around that particular issue.

amitkumars1985 commented 7 years ago

We have globally more than 100 servers serving VNC sessions to users. It's really difficult to plan the upgrade globally. Anyways I will try to test it using both methods on our test server :

  1. By Installing VNC server 2.1.2
  2. VNC server as 2.1.1 and Java Client as 2.1.2
amitkumars1985 commented 7 years ago

After downloading jnlp file from server. It seems to have some missing tags :

server1.net:4 $DISPLAY $PARAMS So jnlp and application-desc tag are missing at bottom of file. After adding those manually, everything works.
amitkumars1985 commented 7 years ago

Sorry..!! Tags missed in my last comment : It's ending like this :

<application-desc main-class="com.turbovnc.vncviewer.VncViewer">
    <argument>server1.net.net:22</argument>

  <

it's missing

</application-desc>

Error in Java Console :

WARNING: the jnlp file is invalid and will be blocked in a future release.
Exception: org.xml.sax.SAXParseException; lineNumber: 73; columnNumber: 5; The element type "application-desc" must be terminated by the matching end-tag "</application-desc>".

Due to which users are unable to launch application.

dcommander commented 7 years ago

Regarding the tags, that must have been an error introduced by your customizations. The default version of the file works fine.

Regarding your server upgrade, your previous communications implied that you were still either evaluating or rolling out TurboVNC. Please contact me (offline, if you prefer) with information regarding who you work for.

amitkumars1985 commented 7 years ago

There are no customization done at our end. We are using pretty much same software which is installed with below changes in turbovncserver-security.conf enable-user-acl pam-service-name = turbovnc permitted-security-types = TLSPlain, X509Plain These setting are changed to enable NIS based authentication to VNC sessions. Other than that nothing else has changed.

Regarding your second point, we have already rolled out 2.1.1 into production 2 months back.

amitkumars1985 commented 7 years ago

and this missing tag issue is occuring only with 2.1.2. On other servers where we have 2.1.1 running. we didn't observe this issue.

dcommander commented 7 years ago

OK, well then please explain how it is possible that there is a missing tag when this is the diff between 2.1.1 and the current tip of the master branch:

diff --git a/java/com/turbovnc/vncviewer/VncViewer.jnlp.in b/java/com/turbovnc/vncviewer/VncViewer.jnlp.in
index 4bb523a..95f9787 100644
--- a/java/com/turbovnc/vncviewer/VncViewer.jnlp.in
+++ b/java/com/turbovnc/vncviewer/VncViewer.jnlp.in
@@ -22,30 +22,50 @@
     <vendor>The VirtualGL Project</vendor>
   </information>

+  <offline-allowed/>
+
+  <update check="always" policy="always"/>
+
   <resources>
-    <j2se version="1.6+" java-vm-args="-server -Dsun.java2d.d3d=false" />
     <jar href="VncViewer.jar" main="true"/>
   </resources>

 @TVNCJNIOPENCOMMENT@
   <resources os="Mac OS X">
+    <j2se version="1.6+" java-vm-args="-server" />
     <nativelib href="ljtosx.jar"/>
   </resources>

   <resources os="Windows" arch="x86">
+    <j2se version="1.6+" java-vm-args="-Dsun.java2d.d3d=false" />
     <nativelib href="ljtwin32.jar"/>
+    <!-- Uncomment if you have the TurboVNC Helper JAR for 32-bit Windows
+    <nativelib href="tvnchelper-win32.jar"/>
+    -->
   </resources>

   <resources os="Windows" arch="amd64">
+    <j2se version="1.6+" java-vm-args="-Dsun.java2d.d3d=false" />
     <nativelib href="ljtwin64.jar"/>
+    <!-- Uncomment if you have the TurboVNC Helper JAR for 64-bit Windows
+    <nativelib href="tvnchelper-win64.jar"/>
+    -->
   </resources>

   <resources os="Linux" arch="i386">
+    <j2se version="1.6+" java-vm-args="-server" />
     <nativelib href="ljtlinux32.jar"/>
+    <!-- Uncomment if you have the TurboVNC Helper JAR for 32-bit Linux
+    <nativelib href="tvnchelper-linux32.jar"/>
+    -->
   </resources>

   <resources os="Linux" arch="amd64">
+    <j2se version="1.6+" />
     <nativelib href="ljtlinux64.jar"/>
+    <!-- Uncomment if you have the TurboVNC Helper JAR for 64-bit Linux
+    <nativelib href="tvnchelper-linux64.jar"/>
+    -->
   </resources>
 @TVNCJNICLOSECOMMENT@

Please point out the missing tag. I don't see it. I just tried the 2.1.2 release candidate build, and it works fine. I also just tried the 2.1.2 release candidate build with the helpers included. Also works fine. Obviously you are using a custom VncViewer.jnlp file, since you are using the TurboVNC Helpers. If you're installing TurboVNC with the official RPM, then the 2.1.2 RPM won't overwrite your /opt/TurboVNC/java/VncViewer.jnlp file. Thus, you probably introduced a bug in that file, and it wasn't fixed by upgrading to 2.1.2 because the file wasn't overwritten with the default version.

dcommander commented 7 years ago

When you uncommented the helpers, did you remove both sides of the comment? I suspect not.

amitkumars1985 commented 7 years ago

I am attaching my jnlp for for you to verify. I don't see any problem in jnlp which is existing on server but when we are trying to download jnlp at client end through web start, it has some missing tags. Both are attached for you.

VNCViewer.zip

dcommander commented 7 years ago

As I said above, you failed to remove both sides of the comment. Look, I appreciate the testing. You have uncovered a few legitimate bugs, and I'm always happy to fix those for free, but I can't provide individual support for free. The free community support model is predicated on users acting as their own first tier of support and sending me only confirmed bugs, along with a procedure that maximizes the odds of me reproducing those bugs in a timely manner. That is predicated on sysadmins such as yourself being willing to dig into the product and diagnose things like improper XML syntax, etc., rather than expecting me to perform that first-tier diagnosis. I'm happy to act as your first-tier support for a reasonable hourly fee, but if I provided that level of support to all TurboVNC users for free, I'd go broke very quickly and this product would die. In the future, I will engage with you on GitHub only if I can readily reproduce an issue you are reporting. If you want support from others in the community, feel free to use the TurboVNC-Users mailing list. If you want individual support from me, your company is going to have to pay for my time.

amitkumars1985 commented 7 years ago

Sorry to bother you again but I really don't understand what you need to tell about removing both sides of the comment. What I have done is as below : Original lines : `

`

Modified lines : `

` and I really don't see any issue in the xml. I would appreciate if you have be bit specific about what is wrong.

dcommander commented 7 years ago

Google XML and JNLP syntax.

amitkumars1985 commented 7 years ago

Ctrl+Alt Key combinations are still not working with 2.1.2. It is still triggering additional KeyPress Event of Shift. Please see the logs below :

` Key Pressed : Ctrl+Alt+j

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269861083, (-491,-340), root:(515,139), state 0x0, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269861255, (-491,-340), root:(515,139), state 0x4, keycode 64 (keysym 0xffe9, Alt_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269861465, (-491,-340), root:(515,139), state 0xc, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269861465, (-491,-340), root:(515,139), state 0xd, keycode 44 (keysym 0x4a, J), same_screen YES,

Key Pressed : Ctrl+j

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269865989, (-491,-340), root:(515,139), state 0x0, keycode 37 (keysym 0xffe3, Control_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269865990, (-491,-340), root:(515,139), state 0x4, keycode 44 (keysym 0x6a, j), same_screen YES,

Key Pressed : Alt+j

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269866744, (-491,-340), root:(515,139), state 0x0, keycode 64 (keysym 0xffe9, Alt_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269866838, (-491,-340), root:(515,139), state 0x8, keycode 44 (keysym 0x6a, j), same_screen YES,

--

Key Pressed : Shift+j

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269868376, (-491,-340), root:(515,139), state 0x0, keycode 50 (keysym 0xffe1, Shift_L), same_screen YES,

KeyPress event, serial 33, synthetic NO, window 0x3c00001, root 0x69, subw 0x0, time 3269868593, (-491,-340), root:(515,139), state 0x1, keycode 44 (keysym 0x4a, J), same_screen YES,

`

If we are pressing only Ctrl or Alt then no additional events triggered. It is happening only with Ctrl+Alt.

g1patnaik commented 7 years ago

Yes, this issue is there for me too..Ctrl+Alt key also adds Shift key event.

dcommander commented 7 years ago

@amitkumars1985 As I explained above:

ivellioscolin commented 6 years ago

2.1.2 has issue on ctrl+shift+f. ctrl+f >>> ok. shift+f >>> ok. ctrl+shift+f >>>NOT ok. ctrl+shift+t >>> ok.

However if you press and hold ctrl, shift, and press other key, like 'a', first, then press 'f' >>> ctrl-shift-f sent.

Update 1: Related to VK_PROCESSKEY. Trying to find a solution.