ESC key not recognized

[garabik]

I am using AVNC to connect to a remote Linux desktop., connecting from a device with a hardware built-in keyboard (Cosmo) and pressing the ESC key ends AVNC immediately. I can reconnect, but using a desktop without ESC is sometimes difficult.

I understand ESC is "special" in Android and you have to code around it - this is something that e.g. Termux does and offers an option to capture and pass the key to the terminal.

Thanks for the otherwise great vnc client

[gujjwal00]

Hi @garabik , I have an external keyboard where ESC key works correctly with AVNC. So it may be something special about your hardware causing this issue. I will need some more info about this:

• pressing the ESC key ends AVNC immediately.

  Is the app crashing or the session is simply disconnected?

• Please go to Settings > Input > Virtual Keys and enable 'Show all keys'. Now reconnect to the server, open Virtual Keys (middle button in Toolbar), and try the Esc virtual key. Does it work correctly?

• Please go to Settings > Tools > Debugging. Open Logs & Keycode Map each and copy-paste their contents here.

[garabik]

Hi @garabik , I have an external keyboard where ESC key works correctly with AVNC. So it may be something special about your hardware causing this issue. I will need some more info about this:

• pressing the ESC key ends AVNC immediately.

  Is the app crashing or the session is simply disconnected?

Disconnected - it returns to the server list. Though I'd swear I saw the app simply disappear before, but that might have been hitting ESC twice or something (in any case, no Android dialog ever appeared).

• Please go to Settings > Input > Virtual Keys and enable 'Show all keys'. Now reconnect to the server, open Virtual Keys (middle button in Toolbar), and try the Esc virtual key. Does it work correctly?

Yes, no problem with a virtual ESC.

• Please go to Settings > Tools > Debugging. Open Logs & Keycode Map each and copy-paste their contents here.

Done. The log was produced while pressing a single ESC key, no other events.

avnc-log.txt avnc-keymap.txt

[gujjwal00]

Logs look normal! As you suspected, ESC key appears to be treated as back-press. Let me look around a bit.

[gujjwal00]

Can you please test the APK I am attaching here? (BTW mentioning Termux was super helpful)

avnc-debug.zip

[garabik]

Anything in the settings I have to change?

No improvement, ESC still disconnects, keyboard map is the same and the log looks the same, too...

[gujjwal00]

No, there was no setting to change. I tried to capture escape key via onKeyPreIme(), which unfortunately didn't seem to work.

I am attaching a new APK. It will log all key events received from Android. Can you please try again (and post the logs this time)?

app-debug.zip

[garabik]

I am attaching a new APK. It will log all key events received from Android. Can you please try again (and post the logs this time)?

Which permissions should I give to the app? I am attaching the log, but there are no obvious events recorded, and the third line looks like a relevant error... avnc-log2.txt

[gujjwal00]

Which permissions should I give to the app?

No special permission is needed here.

I am attaching the log, but there are no obvious events recorded,

Yeah, there is no key event in the logs. That is weird. Is your Android version handling the Escape key itself, rather than delivering it to AVNC? Does Escape key works perfectly with Termux?

and the third line looks like a relevant error...

That doesn't appear to be related to AVNC.

[garabik]

Yeah, there is no key event in the logs. That is weird. Is your Android version handling the Escape key itself, rather than delivering it to AVNC?

Cosmo is known for its weird keyboard handling but from the description it does not seem to affect the ESC key (unless the App bar is open, which is not relevant here).

Does Escape key works perfectly with Termux?

Yes. Of course, only with back-key=escape in .termux/termux.properties

[gujjwal00]

Looks like it does affect the Esc key, otherwise you wouldn't have to specify back-key=escape in Termux (which simulates Escape on Back press). Although in the last logs we didn't even receive a Back key event.

Can you pull /system/framework/services[.jar] (as noted in your article) using adb, and link it here? I want to see exactly what they are doing with Escape, and if there is any workaround.

[voidbert]

Hey! I started using AVNC recently and I'm experiencing similar behavior. In my case, I am sure my Android Launcher (or Android System itself) is handling the event and taking me to the Home screen. It's not handled the same way as the back key, as that just closes the VNC activity and takes me to the place to choose a server. This isn't an annoyance for my use case but I think it can be useful information for this issue, thus I'm sharing it.

EDIT: It happens with all applications

[gujjwal00]

Hi @voidbert, thanking for sharing.

If you want, you can try the APK from this comment: https://github.com/gujjwal00/avnc/issues/12#issuecomment-957316034.

Unfortunately this type of issues are tricky to fix as I don't have a similar device to test on, and there is no documentation about the behavior. And I know this is frustrating to reporters too, as they have to test a bunch of different approaches.

[garabik]

Can you pull /system/framework/services[.jar] (as noted in your article) using adb, and link it here? I want to see exactly what they are doing with Escape, and if there is any workaround.

Attached - though it seems to be empty. I am also attaching /system/framework/oat/arm64/services.{art,odex,vdex}, these might be the relevant files (note I am not very familiar with Android). services.jar.gz services.art.gz services.odex.gz services.vdex.gz

[gujjwal00]

I have gone through these files. Most of the tools fail to de-compile the relevant methods, but whatever information I could gather seems to suggest that PhoneWindowManager class is not translating Escape to Back key. I have even downloaded the full System Image for Cosmo Communicator and de-compiled a bunch of different vdex files, hoping to locate where Escape is being handled, but haven't found anything yet.

So it seems like only solution (for now) is to simulate Escape when we receive back-press. To do this we need to know exactly how this device is finishing the VncActivity (i.e. disconnecting the session) when Escape is pressed. I am attaching another APK here which have some more verbose logging.

app-debug.zip

When you have some time, please test it out:

1. Clear any previous logs
2. Connect to the server. Press A (or any other character). Now press Esc to disconnect.
3. Reconnect to server. This time touch the Back ◂ button to disconnect.
4. Grab the logs.

[garabik]

1. Clear any previous logs
2. Connect to the server. Press A (or any other character). Now press Esc to disconnect.
3. Reconnect to server. This time touch the Back ◂ button to disconnect.
4. Grab the logs.

Attaching - I pressed A, ESC; then reconnected and rpessed B and Back button

avnc-log3.txt

[gujjwal00]

These logs seems incomplete (there are no KeyEvent logged). There should be a lot more info there (all the keys you pressed, multiple stack-traces etc).

Please try getting logs using adb logcat. It is also possible that the system is stripping all debug logs, and only printing warning/errors.

[gujjwal00]

For example, this is what logs look like on my emulator: log.txt

[garabik]

Please try getting logs using adb logcat. It is also possible that the system is stripping all debug logs, and only printing warning/errors.

Done. There are some key events, hopefully you will be able to make something out of if. The sequence of events was Connect, press a, press ESC, reconnect, press b, press the Back button. logcat.txt.gz

In the meantime I've got an external bluetooth keyboard, and this one behaves differently - ESC does not disconnect, but returns to the launcher, with AVNC connection still running PIP. Cosmo definitely does something behind the scene with the built in keyboard events.

[gujjwal00]

That was really helpful. Everything makes sense now. I can even follow the de-compiled services.vdex, and figure out exactly what is happening.

So, pressing Esc generates neither KEYCODE_ESCAPE nor KEYCODE_BACK. It generates freaking KEYCODE_POWER. Then PhoneWindowManager (from services.vdex) intercepts this KEYCODE_POWER and tests a bunch of flags:

• If a call is ringing, silence it
• (may be) end an ongoing call
• (may be) handle multi-press actions
• some other stuff
• and if nothing else to do, simulate a KEYCODE_BACK

This is the relevant portion of the logs (keyCode = 26 is KEYCODE_POWER, and keyCode = 4 is KEYCODE_BACK ):

11-11 11:36:55.972   911  1139 D InputReader: GUOHUAJUN updateMetaStateIfNeeded keyCode=26 ,mMetaState=0,down = 1
11-11 11:36:55.972   911  1139 D zengtao : interceptKeyBeforeQueueing keyCode = 26~~~~~~ down = true
11-11 11:36:55.972   911  1139 D WindowManager: interceptKeyTq keycode=26 interactive=true keyguardActive=false policyFlags=22000000
11-11 11:36:56.050   911  1139 D InputReader: GUOHUAJUN updateMetaStateIfNeeded keyCode=26 ,mMetaState=0,down = 0
11-11 11:36:56.050   911  1139 D zengtao : interceptKeyBeforeQueueing keyCode = 26~~~~~~ down = false
11-11 11:36:56.050   911  1139 D WindowManager: interceptKeyTq keycode=26 interactive=true keyguardActive=false policyFlags=22000000
11-11 11:36:56.050   911  1139 D WindowManager: powerPress: eventTime=256006113 interactive=true count=1 beganFromNonInteractive=false mShortPressOnPowerBehavior=1
11-11 11:36:56.050   911  1139 D WindowManager: Sending Back
11-11 11:36:56.053   911 32057 D zengtao : interceptKeyBeforeQueueing keyCode = 4~~~~~~ down = true
11-11 11:36:56.053   911 32057 D WindowManager: escape key
11-11 11:36:56.053   911 32057 D WindowManager: interceptKeyTq keycode=4 interactive=true keyguardActive=false policyFlags=2b000000
11-11 11:36:56.055   911 32057 D zengtao : interceptKeyBeforeQueueing keyCode = 4~~~~~~ down = false
11-11 11:36:56.056 31281 31281 D FrameView: onKeyPreIme: KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=256006191, downTime=256006113, deviceId=-1, source=0x0 }
11-11 11:36:56.056   911 32057 D WindowManager: escape key
11-11 11:36:56.056   911 32057 D WindowManager: interceptKeyTq keycode=4 interactive=true keyguardActive=false policyFlags=2b000000
11-11 11:36:56.057 31281 31281 D FrameView: onKeyDown: KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=256006191, downTime=256006113, deviceId=-1, source=0x0 }
11-11 11:36:56.057 31281 31281 D VncActivity: onKeyDown: KeyEvent { action=ACTION_DOWN, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=256006191, downTime=256006113, deviceId=-1, source=0x0 }
11-11 11:36:56.059 31281 31281 D FrameView: onKeyPreIme: KeyEvent { action=ACTION_UP, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x0, repeatCount=0, eventTime=256006191, downTime=256006113, deviceId=-1, source=0x0 }
11-11 11:36:56.060 31281 31281 D VncActivity: onKeyUp: KeyEvent { action=ACTION_UP, keyCode=KEYCODE_BACK, scanCode=0, metaState=0, flags=0x200, repeatCount=0, eventTime=256006191, downTime=256006113, deviceId=-1, source=0x0 }

Solution: We have to simulate KEYCODE_ESCAPE when we receive KEYCODE_BACK. I am attaching a new APK with required solution. Hopefully this is the last one you will have to test 🤞. It also tries to detect whether KEYCODE_BACK is coming from navigation bar or somewhere else, so touching the Back ◂ button should disconnect the session normally. app-debug.zip

Now, all I am wondering is why the hell are they doing this. Even if they had to generate KEYCODE_POWER, they could have simulated KEYCODE_ESCAPE, instead of KEYCODE_BACK. Android will fallback to KEYCODE_BACK if application does not handle KEYCODE_ESCAPE. It does that by default!

You mentioned that other keyboard's Esc key behaves differently. If you are returning to launcher, with AVNC in PiP, then that Esc key is probably generating KEYCODE_HOME, which is not even delivered to apps. If you want, you can paste the logs (using that keyboard) here and I can try, otherwise no problem.

[garabik]

So, pressing Esc generates neither KEYCODE_ESCAPE nor KEYCODE_BACK. It generates freaking KEYCODE_POWER.

Actually, this makes some sense - on Cosmo, Fn+ESC is an equivalent of POWER button, it turns off the screen (and locks it and puts into a sleep mode eventually), and hoding ESC turns the power on (or brings up poweroff dialogue). Obviously, they hardwired ESC as the Power button and Fn is the answer to the multi-press action you saw. Of course, it could have been done differently/better. Well...

Solution: We have to simulate KEYCODE_ESCAPE when we receive KEYCODE_BACK. I am attaching a new APK with required solution. Hopefully this is the last one you will have to test 🤞. It also tries to detect whether KEYCODE_BACK is coming from navigation bar or somewhere else, so touching the Back ◂ button should disconnect the session normally.

It works perfectly - ESC is passed as ESC, Back button disconnects.

You mentioned that other keyboard's Esc key behaves differently. If you are returning to launcher, with AVNC in PiP, then that Esc key is probably generating KEYCODE_HOME, which is not even delivered to apps. If you want, you can paste the logs (using that keyboard) here and I can try, otherwise no problem.

I am not going to use the keyboard with my Cosmo - after all, I have a good built in one. And that BT keyboard has a really crappy layout. But it was quite cheap, it is small and light and backlit and mechanicaly quite good, all things considered. So I might do some tests, after all, keyboards generating KEYCODE_HOME on ESC are probably common.

Anyway, thank you very much for your effort - you really went out of your way in debugging this.

[gujjwal00]

Fixed in https://github.com/gujjwal00/avnc/commit/7400a38c36a66665a6b7ae16bc15dc4ec2fd10c7

To avoid any unexpected issues with the current key handling, I am putting this feature behind a preference Settings => Input => Key Mappings => Back to Escape, which is disabled by default.

It will be available in the next release.