Huavei 10 mate lite Capslock state detection problem

[JanuszChmiel]

Dear specialists, I Am using yours app with Archlinux Aarch 64 which run by using Termux and Android device. Sure, Multivnc run on normal Android phone. I have issues with The Capslock key detection state. Orca screen reader uses XKB protocol to get information about pressed keys. For example. When I turn Capslock on, I can correctly type big letters. But Orca do not detect if Capslock have been pressed and its state. On or off. This problem is independent which Hardware keyboard model will I use. Orca can not also correctly detect if I have pressed left ALT or right ALT key. Orca speak left ALT even I press right ALT. The most annoing issue is The fact, that I can not control Orca navigation system. Because if Orca do not detect CAPSLOCK state, it can not react on key combination CAPSLOCk and some letter such as u, j or l. Do you think, that there is some chance to find out The ccause of this issue? The problem is really in VNC client app. May be, that Orca need to get more specific more exact info about keys. The question is, if is itpossible to develop such code inside yours app, which would inform this screen reader about The state of CAPSLOCk key. Sure. Orca have been developed for A situation, where user uses it iwth standard Linux not controlled by using VNC protocol. Any help will be very welcomed. Thank you very much for yours time.

[JanuszChmiel]

Orca can correctly detect. Left or right CTRL keys. ALT key can not be detected correctly. Orca present left ALT key even if I press Right ALT key. Numeric Insert can not be detected too. Other keys are working perfectly. I have also tested External USB hardware keyboard. What do you think, what is The cause of those issues? Specific Orca app, which need too much specific information. Or VNC protocol, which can not because of its nature transfer all needed info from running Xvnc to Android client?

[boroli]

Same problem here on my Huawei MediaPad M3 Lite 10. The Shift key is no longer recognised within MultiVNC when connected to VNC server from a debian running in a termux environment on localhost. Which makes it pretty hard to use in the end... Apparently it worked flawlessly in the past (~6 months ago I would guess). Unfortunately, I can't retrace with which update this was introduced. Shift key works in Android without any problem including the termux terminal.

[bk138]

I can provide you with older versions to test, if you want. Which one are you using right now?

[boroli]

Sure. Happy to test. At the moment 1.9.5 from F-droid. Already tried to downgrade from the most recent one, but without any success.

[bk138]

I propose you start with 1.8.12 from https://apkpure.com/multivnc/com.coboltforge.dontmind.multivnc/versions and go back if the issue is there as well.

[bk138]

@boroli actually, are you sure this is the right issue? @JanuszChmiel was reporting Caps and Alt, you're reporting Shift.

[boroli]

Well, I don't know (and can't at all) if it's the same bug, but it sounded pretty close. Caps Lock didn't work for me either, and I didn't test ALT. I tried 1.8.12 and at least Shift is working again properly. Also Caps Lock is working at least for characters, but not for numbers of the first line. Ctrl and Alt keys work partially, but not fully as one would expect from desktop input. For example in Spyder3 IDE, Ctrl+z is working, whereas Ctrl+1 is doing nothing. Apparently this (Ctrl+1) didn't work in the past either. So I'm not sure if this is an Android limitation itself, or one from MultiVNC.

[bk138]

Please also have a try with 1.9.4 attached here and try "native" vs "traditional" mode: app-debug.zip

[JanuszChmiel]

Thank you very much for yours programmers work. I will try this .apk file on my mobile phone. To be honest, supporting Orca screen reader build in keyboard shortcuts detection by using VNC protocol is standalone not easy programmers task. Because Orca is being developed with The assumption, that user will run this screen reading software by using normal Linux os, non virtualized and not controlled by using VNC protocol. As A result, there are many complex semantic programmers issues which would had to be solved only if somebody very patient and advanced would have some spare time to analyse routines build in to Orca Python scripts, and on its complex dependent libraryes, which are being used for scanning keyboard codes. There is also very complex programmers problem. Because Android keyboard system app detect some key stroke and there is non easy task how to notice Android OS that some other app want to receive keyboard control as soon as possible. TO be honest, I have tried many many solutions including Xserver XSDL, many commercial and free VNC clients. And no VNC clients are able to 100 % reliably support keyboard controls for Orca. For example. It is ery complex to develop VNC client which can reliably detect, if user has pressed left or right Shift key, left or Right CTRL key, ETC. In this cases, Orca report left and right key very soon and it do not report left or right if user hold standalone Shift or CTRL for a longer time. The programmers question is, if it is programatically possible reliably detect key strokes by using build in techniques for Android programming. Because even developers, who are using native C or C++ libraries for such tasks have not a guaranteed, that his VNC client will work 100 % reliably with Orca. Thank you again and I will let you know The esults. Now I Am using Tigervnc not X11vnc and Xorg. Because Tigervnc support combination left SHIFT and TAB more efficiently. Because when I have used X11vnc app, I have not been able to use focus navigation. When I have pressed SHIFT+TAB X11 APPS simply ignored this key stroke and app have behaves like I have simply only pressed TAB key. So I could not navigate by using SHIFT+TAB keys.

-- Tento e-mail byl zkontrolován na viry programem AVG. http://www.avg.cz

[JanuszChmiel]

I must confirm, that Shift key Capslock key and other special keys are being correctly scanned by yours app. The only one problem is, that build in Orca screen reader condition blocks which are responsible for pronouncing, which key have been pressed are not able to detect The names of Shift and Capslock. By other words. Other X11 apps can react on combination left Shift+TAB correctly. Even if I press Capslock on my external BLuetooth keyboard, Orca do not report that I have pressed Capslock key, but Orca report correctly, that I AM typing by using big letters. When I turn Capslock off Orca pronounces that I Am typing small letters when I type letters on A keyboard. I will try to find out, if Capslock+Space will recall Orca build in preferences dialog. If no, there is some complex and hydden issue. I do not know how to switch between native or traditional mode by using yours app. If this mode can be switched after X11 connection so VNC window show bitmap, it is problem, since Explore by touch feature of Talkback can not detect such button. I have positive notice. Orca perfectly recognize if I have pressed left or right CTRL. And CTRL key is being pronounced when I have pressed it. Shift key is not being pronounced, but it react when I Am typing with Shift. So very probably Orca build in keyboard tutor default active Orca feature have been programmed for A situations, when Keyboard have been detected by using UDEV. Orca programmers did not assumed, that somebody will use it with VNC protocol. But it is possible to tune Multivnc, but it would require some patience and time. It will depend on yours good will. I will now press CAPSLOCK+Space to find out, what will happen.

-- Tento e-mail byl zkontrolován na viry programem AVG. http://www.avg.cz

[JanuszChmiel]

Orca can not recall preferences dialog when Capslock+Space have been pressed. Because Orca did not detected Capslock. It is very strange situation. Orca build in keyboard typing algorithm scanning correctly recognizes, if I AM typing big or small letter after I have pressed Shift or Capslock. But unfortunately, Hod key with Capslock do not work.

-- Tento e-mail byl zkontrolován na viry programem AVG. http://www.avg.cz

[boroli]

Please also have a try with 1.9.4 attached here and try "native" vs "traditional" mode: app-debug.zip

No improvement to 1.8.12 for traditional mode. Native mode is even worse. Shift and CapsLock doesn't work.

[bk138]

Can you please try to make a table or listing where shift works and where not? That'd be really helpful in pinpointing this down.

Am 30. November 2020 05:11:39 MEZ schrieb boroli notifications@github.com:

Please also have a try with 1.9.4 attached here and try "native" vs "traditional" mode:

app-debug.zip

No improvement to 1.8.12 for traditional mode. Native mode is even worse. Shift and CapsLock doesn't work.

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/bk138/multivnc/issues/68#issuecomment-735533501

-- Diese Nachricht wurde von meinem Android-Gerät mit K-9 Mail gesendet.

[boroli]

Status of the Shift key: 1.8.12 (apkpure) yes 1.9.4 Traditional (app-debug.zip) yes 1.9.4 Native (app-debug.zip) no 1.9.5 (F-Droid) no 1.9.7 (F-Droid) no

[bk138]

Hi guys, was this closed by 03770d98d01c061ee8ab61ea77c0d500f7c9b6fb (which is in version 1.9.10)?

[bk138]

@JanuszChmiel @boroli friendly ping about this Huavei 10 mate lite Capslock state detection problem issue :-)

[JanuszChmiel]

When I have tried .apk several months ago, Capslock have not been correctly detected. BVNC Pro have solved my problems with Capslock. Proot developers have even implemented SHMGET and SHMAD so shared memory work as it should.

[bk138]

When I have tried .apk several months ago, Capslock have not been correctly detected. BVNC Pro have solved my problems with Capslock. Proot developers have even implemented SHMGET and SHMAD so shared memory work as it should.

Would you mind giving it a quick try with MultiVNC 1.9.11 for the sake of testing @JanuszChmiel ? That would help us a lot!

[JanuszChmiel]

Yes, with big pleasure. I must only install Termux and ArchLinux, because I have given it up after several months. But I Think, that I will do it during this day. Thank you for yours programmers work. Orca screen reader is very complex app and if Capslock will not be detected by it, somebody of yours team would had to have some big patience and somebody would had to study dependent Python library for scanning hot keys to find The cause and I do not know, if it would not be too much for us as A kind volunteers. To be honest, orca screen reader is suffering with very complex problém which is also very annoing. Very probably because none of Orca screen reader developershave thought, that some users will use Orca outside of Standard Linux environment. So Orca screen reader support two keyboard layouts. Desktop or Laptop. When visually impaired Orca user want to use WEB browsers, Orca programmers have implemented special quick navigation keys to quickly jump to all supported WEB browser object such as buttons, radio buttons, ETC. Unfortunately, when user uses some Android VNC client and VNC protocol, Orca detect keys from 1 to 9 only as a numbers but do not detect those keys as quick navigation keys to jump to heading number 1 to 9. No matter, if I choose Czech or US ENglish layout. And solving such complex and specific issues is not easy and I do not know how to help constructively to solve them. I will begin by testing yours provided .apk package.

[bk138]

@JanuszChmiel you can use this apk.

[JanuszChmiel]

Please send direct .apk package download link.

[bk138]

Please send direct .apk package download link.

https://f-droid.org/repo/com.coboltforge.dontmind.multivnc_79.apk

[JanuszChmiel]

Thank you very much. I have just downloaded The .apk package.

[JanuszChmiel]

I have just started to install ArchLinux and I will install The necessary components For quick testing, I will only install Tigervnc, Orca crucial speech components and some window manager, Marco and Orca. I will also install pluma for example. To have my testing finished as quick as possible for you. Fortunately for Me, I have done notes during my previous activities with Termux, ncluding packages names and other thinks such as command line for starting Tigervnc without password. I have MultiVNC allready installed, thank you for package link. I will let you know as soon as possible. My Huavei 10 Mate Lite with Android 8.0 and with 8 core CPU is doing his best to work. I would like to notice, that it is always A good practice to force Android OS to run Termux and Multivnc or other VNC client in background, in other case, in A long time of inaktivity, Android can suspend or terminate The proces. When I Am using battery and app manager and I will force systém to run app in background, VNC client can listen for internal TCP/IP communication even for A whole night. It is even possible to let Mate desktop run for A whole night while having Android evice screen turned off. Sure. Every one, who love Linux and GTK or other X11 apps, that it is not Android so there are no functioning battery optimisations routines. So you can not anticipate long battery life. But it is not disaster, because battery can not be discharged and there is no main board overheat, because when user do not execute Linux kernel directly by using telephone CPU, there is no difficulty. Dangerous exeriment would be, if amateour user would try to execute AARCH64 Linux kernel which have been compiled by non professional, who would not setup kernel compilation to prevent main boare overburning, etc.

This is The reason, why specialised phone manufacturers are manufacturing phones with preconfigured Linux kernels with included kernel drivers for chip. From The security perspective, Proot is The best solution, because systém calls are being used and Android kernel have The main control on The running device. So there are no dangerous incidents so easily possible.

[JanuszChmiel]

I have a bad news. Orca contain routine, which is able to inform user which key user pressed. For example. Left CTRL , right Shift. Unfortunately, this routines are not able to speak that user has pressed left Shift and even Capslock. Now I Am working with one USB hardware keyboard connected to Micro USB by using OTG. But The same problém exist also if I connect Bluetooth hardware keyboard. The problém is, that for example, other GTK apps are able to detect all key strokes and keys. Very complex issue, which have very probably something to do with The fact, that Orca Python libraryes, which have been created for full Linux operating systém do not support VNC clients. FOr Android. Or Android OS have some internal limitations. BVNC support Capslock. But how to debug this problém? It is may bee too complex.

[JanuszChmiel]

But other GTK processes and X11 modules are correctly detecting all keys. Because when I press Capslock, letters on my keyboard are being typed as big letters. Orca screen reader speak by using arrow keys. By other words, Only orca routines have problems with keys so sighted majority will not observe this issue which is only specific to Orca. So you can close this issue.

[JanuszChmiel]

And I have positive info. I can use even touch keyboard, Android accessibility suite perfectly speak keys so even when VNC session work, I can use touch glass. I hear keys from Android screen reader and when I leave The key, I hear its name by using Orca. I will try The touch keyboard Capslock what will happen. Other VNC clients do not allow screen readers users to use finger to hear pressed keys. CLient is very fast so Shift key is not detected by routine for informing users about pressed keys, but it do what it should. When I type with Gedit app, letter is really big.

[gujjwal00]

I think we are missing conversion from KeyEvent.KEYCODE_CAPS_LOCK to X key-symbol XK_Caps_Lock (0xffe5) in here: https://github.com/bk138/multivnc/blob/d4d3b8421c921e038655e9c96a8b95d3043182d6/android/app/src/main/java/com/coboltforge/dontmind/multivnc/VNCConn.java#L584-L585

This would fix MultiVNC but Vino simply ignores Caps-lock events (don't know about Tigervnc).

[bk138]

I think we are missing conversion from KeyEvent.KEYCODE_CAPS_LOCK to X key-symbol XK_Caps_Lock (0xffe5) in here:

https://github.com/bk138/multivnc/blob/d4d3b8421c921e038655e9c96a8b95d3043182d6/android/app/src/main/java/com/coboltforge/dontmind/multivnc/VNCConn.java#L584-L585

This would fix MultiVNC but Vino simply ignores Caps-lock events (don't know about Tigervnc).

I can have a look.

[JanuszChmiel]

I must confirm, that there is An very complex relationship between .Tigervnc and other VNC protocol based app. Problem is, that advanced Multivnc C++ programmers can not debug such non standard conditions like strictly userspace based variant of Linux distro which run on The top of non rooted Android device.

BVNC Pro and BVNC Free allow Orca screen reader to get such events. But also there is some complex semantic issue. When user hold left or Right shift, event is detected according to A user hold delay to this key. To exclude that it is problem caused by Android AOSP or other similar module, I must find some touch keyboard which contain Capslock key, not only Shift key. But may be, that such keyboard even do not exist at all. I will also try Blind accessibility keyboard with Multivnc. It allow Me to press every letter twice to write it. Other keyboards write letter when user lift his finger. Samsung keyboard contain this choice but not other device manufacturers are including similar keyboard.