[Android 11] Mouse click doesn't work on my Google TV unless target item has focus

[fil-dariane]

Hi virresh,

Many thanks for your work and for the way you handle this project.

I have a new 2021 TCL Series 6 TV running Google TV based on Android 11. I've installed MATVT v1.0.6 successfully and can see and move the mouse cursor. But pressing the center button to click the mouse does not work -- unless the item I'm clicking already has focus, i.e. shows the visible indication of focus and would normally respond if you press the center button in normal DPAD mode. It also seems like the DPAD buttons have no effect in Scroll Mode. A normal mouse plugged into the USB port of my TV works as expected.

Please let me know if there is any workaround I might try, and thank you for your attention.

[virresh]

Thanks for reporting @fil-dariane

It has come to my knowledge that several TV providers are removing the gesture APIs from their Operating systems. MATVT relies on this API for clicking on places where otherwise a click is not feasible.

As you mentioned it works when item is focused, this is because of a fallback that I'd coded in for Android 6.

In order to make the mouse work on OSes without support for gesture API, we'll need some root level permissions, or a miraculous work-around.

I'm working on a completely different core engine (details in #24), but as reported by @sweenwolf it'll lag behind the current normal mouse. I'll post a pre-release apk and inform you here. Would need you to test that out when it's available 😄 .

[fil-dariane]

Thanks for your response and explanation @virresh. I can test on my TV when you have something.

[virresh]

@fil-dariane can you test with https://github.com/virresh/matvt/releases/download/v1.06-pre-release/matvt-test-adb-release.apk ?

[fil-dariane]

Hi, @virresh. Here's my testing feedback using my TCL Google TV.

The new MATVT functionality that you have just added can be intermittent on my TV. When this happens, the MATVT mouse click and scroll do not work. However, the MATVT DPAD functionality still works, the TCL stock remote control works, and an external USB mouse works. The MATVT app shows that the permissions are set and the service is running. This problem state can already exist after MATVT installation or it can just happen suddenly later for no apparent reason. Restarting the MATVT service in System -> Accessibility does not fix it. Restarting the TV can fix it.

When the new MATVT functionality is working, the mouse click works fine and the latency is about half a second, which is acceptable. However the scroll functionality has an issue due to slow handling of queued scroll events. Touching a button quickly in scroll mode typically generates 3-6 scroll events. The first one executes a half second later, which is fine, but then there is a regular one-second delay between all the remaining queued events. If you hold the button down longer, it can easily generate 50-100 scroll events. This can make the UI of the TV unusable for more than a minute.

You should be aware that these new TCL Series 6 Google TVs like mine have recently been in the news for OS and latency issues. https://www.theverge.com/2021/12/9/22826619/tcl-6-5-series-google-tv-best-buy-removal-os-bugs-slow I have indeed seen such issues, but I can say that the TCL UI sluggishness is sporadic and only lasts for a second or two, so it is easy to recognize and I can still judge MATVT responsiveness fairly.

[fil-dariane]

Just to be clear, I see no scroll latency issue when the DPAD is used to scroll a window. I can hold the button down in MATVT DPAD mode or on the stock remote and it scrolls fast with no latency at all between scroll events. It's only the MATVT scrolling that has this issue.

[virresh]

Thanks for testing @fil-dariane I'm aware about issues with the scroll queuing up and scroll being slow (basically I didn't get enough time to properly implement it, but I think some things like scroll speed may no longer be possible in the new engine).

The intermittent issue with MATVT glitching out is an interesting one. I haven't come across that yet. My best guess is that adb quietly goes to sleep and requires a force close of the app in order to re-establish the connection. I'll have to do some more investigating on that part. Do share if you get any more info on this.

Unfortunately, the delay between two scroll events is something I cannot control. It's an issue with adb latency as mentioned in the other issue. I can try to manage the queue a bit better and discard all pending scroll events in case some other button is touched.

Thanks for testing it out. I think it'll take a lot more time to tune out the rough edges, but I hope I'll get some time to work on them soon.

[fil-dariane]

Managing the queue to discard excess scroll events should fix that backlog problem, assuming there's a good point during scroll event execution when you have control to do that. Whenever I restart the TV and try to use MATVT, I always get the prompt asking me to "allow USB debugging", even if I check the box to "always allow from this computer". I suppose this is when it sends the ADB commands to set up the needed functionality again.

[fil-dariane]

Just a question: If ADB lib latency is an issue, is there a way to emulate a USB device? I see that a physical mouse plugged into the USB port has no latency issues.

[virresh]

I suppose this is when it sends the ADB commands to set up the needed functionality again.

Yep.

is there a way to emulate a USB device?

A good question, and yes it's possible, but when emulating a USB device, I do not have an option to intercept the existing remote's buttons. I need to have a hook at the software level, and without rooting, this is only possible with an Accessibility App. Here's where the catch lies. Having an accessibility app means I need to be inside JVM, and being inside JVM, I cannot directly interface with the kernel for communicating to my custom drivers. Also in order to successfully install such a driver, one will need root permissions on the Android TV, something normal users will find insanely hard to do. Which is why, we're stuck until Android gives us something (like the gesture API, which the TCL OS devs have readily removed from your device 😭 )

[zdpark]

I have same issue using Shield tv since I updated it to android 11.

[virresh]

@zdpark have you tried the pre-release version above https://github.com/virresh/matvt/issues/28#issuecomment-991885056 ?

[zdpark]

@zdpark have you tried the pre-release version above #28 (comment) ?

Yeah I did. same problems that fil-dariane had. Actually, the click function is not intermittent, it only works on home launcher.

I heard there are some changes with the gesture api since android 11.

[virresh]

I heard there are some changes with the gesture api since android 11.

Do you have any reference for this? If I get to know the exact changes, it might be possible to revive the old engine itself and make it compatible instead of relying on adb hacks. That should eliminate the problem.

[fil-dariane]

Some more feedback re. the "intermittent" issue:

Sometimes when I see this, I notice that another app using Accessibility Service (Button Mapper by flar2) has also stopped working, so you're not the only one. In this case, the apps are shown as Off/Disabled under Settings -> System -> Accessibility. I set them On (and typically Off and On again) and the apps start working again. Perhaps some internal process for Accessibility has restarted unexpectedly.

My TCL Google TV Android System does seem to be underpowered and buggy. I get better and more reliable behavior by keeping the process count down and by doing a full restart every so often. Hopefully future promised updates will help, but it must cause problems if certain processes restart unexpectedly.

There is a setting Settings -> Developer options -> Background process limit, which can be set to Standard, none, 1, 2, 3, or 4. I read that this primarily affects cached services rather than apps. I had been experimenting with this, but to avoid confusion here, I decided to leave it set to the default Standard (supposedly 20?) and to manually terminate unneeded background apps myself.

Another source of confusion is that it can be tricky to confirm the ADB prompt to allow USB debugging when it pops up for MATVT. It occurs at the first click attempt when the need is detected. At that time I need to use DPAD mode to confirm. If I don't confirm it correctly, there will be no chance to do it again until restart, which explains my observed "intermittent" behavior requiring restart.

Separate from this: It is also an issue that I cannot obtain a "long" press by holding the click button down. I suppose the latency of this method makes that difficult.

[juzmafia]

I have same issue using Shield tv since I updated it to android 11.

NVIDIA has responded on this issue and they will add the fix it in the future hotfix 9.0+ release.

@virresh NVIDIA has shared the cause of this issue here: https://www.nvidia.com/en-us/geforce/forums/shield-tv/9/482136/mouse-toggle-app-not-working-on-shield-since-upd/3203693/

Apparently, Google has fixed it in Android 12.

[virresh]

Thanks @fil-dariane for the clarifications. It makes sense that the OS would kick out matvt services from background process to free up ram space, and I don't think there's any way around that. Regarding the ADB prompt, maybe will try to make it a bit easier someday, but given the above comment about Android 11 updates planned to fix this, I'd probably wait before implementing anything on this hacky engine.

@juzmafia Wow, that's huge. I had given up hope on TV manufacturers fixing back the functionality in their Android 11 OSes. Thanks for the forum link!

Hopefully updating to a newer version should fix the problems for most people then.

I'll keep the experimental ADB-driven engine here itself as a pre-release only, just in case anybody needs it. It's not pretty, but does the job 😅

[fil-dariane]

@juzmafia Thanks for sharing this information.

@virresh That seems like the right way to proceed based on what we're hearing so far. My TCL TV just sent me an OS update, but it's still Android 11 and when I retested the official non-ADB MATVT it still doesn't work. I will let you know if I have any further news and use the ADB MATVT in the meantime.

[dikodahan]

Is there any update on this one? I am having the same issues on the Chromecast with Google TV, which is running Google TV 10 & also on a Mecool KM7 box running Android TV 11. I have tried both the original 1.06 and the pre-release ADB version of 1.06. Unfortunately, no luck with either...

[virresh]

@dikodahan chromecast buttons work slightly differently, so you might have to configure your chromecast to send input to Android instead of sending it to TV.

The pre-release ADB version is meant for Android 11 boxes, can you explain what kind of issue you're facing? Are you able to enable remote debugging on your Android TV ?

[ytsystemtv]

Hi, for me it's working on Realme Stick 4k and Mi tv Stick 4k

====== Maintainer Edit Start ======= Please watch in this video: [REDACTED] download link, Video links in below comment

Note: Please prefer downloading from official sources (https://github.com/virresh/matvt/releases), instead of creating mirrors. Mirrors are dangerous because of undisclosed source code and hurt the downloads stat, the only thing that is available to the project author as a metric of project usage. ====== Maintainer Edit End =======

Are you active the developer mode?

Em seg., 21 de mar. de 2022 17:33, Moshe Dahan @.***> escreveu:

Is there any update on this one? I am having the same issues on the Chromecast with Google TV, which is running Google TV 10 & also on a Mecool KM7 box running Android TV 11. I have tried both the original 1.06 and the pre-release ADB version of 1.06. Unfortunately, no luck with either...

— Reply to this email directly, view it on GitHub https://github.com/virresh/matvt/issues/28#issuecomment-1074391955, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYCNETYK4KKIXDOOLDIXCFTVBDMKXANCNFSM5JP4276A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[virresh]

@ytsystemtv I presume good faith, but can you provide the source code for the APK you linked above? It's dangerous to use apks without knowledge of it's source. If it's the same as the one in releases, please do not create mirrors. It only leads to unnecessary confusion.

[dikodahan]

Not the first time I have been using this app (great app BTW!). Even on the Chromecast with Google TV it worked without issues originally. Maybe a recent update to the OS added something that causes it to behave the same as on Android 11 now?

[ytsystemtv]

Sorry, I was going to send the link to my video and I accidentally sent the abbreviated apk link but it's the same one you provided here, it's just shortened the link.

Realme Stick 4k : https://youtu.be/yUOxufPKLHE Mi tv stick 4k: https://youtu.be/bbkeSL0Ljq8

Em sáb., 26 de mar. de 2022 10:09, Viresh Gupta @.***> escreveu:

@ytsystemtv https://github.com/ytsystemtv I presume good faith, but can you provide the source code for the APK you linked above? It's dangerous to use apks without knowledge of it's source. If it's the same as the one in releases https://github.com/virresh/matvt/releases, please do not create mirrors. It only leads to unnecessary confusion.

— Reply to this email directly, view it on GitHub https://github.com/virresh/matvt/issues/28#issuecomment-1079691357, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYCNET4AY3ODOAEVQL4FWSDVB4D73ANCNFSM5JP4276A . You are receiving this because you were mentioned.Message ID: @.***>

[ytsystemtv]

In the two videos I show that I managed to configure the Realme Stick 4k (Google tv) and Mi tv stick 4k (Android tv 11) ... all functions are working very well, except for scrolling, we press once and it multiplies the touch by 4x ... I would like to give you this feedback. Thank you! I'm following the news!

Em sáb., 26 de mar. de 2022 10:09, Viresh Gupta @.***> escreveu:

@ytsystemtv https://github.com/ytsystemtv I presume good faith, but can you provide the source code for the APK you linked above? It's dangerous to use apks without knowledge of it's source. If it's the same as the one in releases https://github.com/virresh/matvt/releases, please do not create mirrors. It only leads to unnecessary confusion.

— Reply to this email directly, view it on GitHub https://github.com/virresh/matvt/issues/28#issuecomment-1079691357, or unsubscribe https://github.com/notifications/unsubscribe-auth/AYCNET4AY3ODOAEVQL4FWSDVB4D73ANCNFSM5JP4276A . You are receiving this because you were mentioned.Message ID: @.***>

[virresh]

Cool. I'm aware of the scroll issues, please check the discussion above.

Maybe a recent update to the OS added something that causes it to behave the same as on Android 11 now?

@dikodahan yep, seems like it. I have no other explanation. Unfortunately, I don't have a Chromecast with me to check this though.

[CrueRock]

Thank you for your work, Virresh New user of Chromecast with Android TV 12 Build STTF.220621.013 and MATVT 1.0.6 If I'm understanding this thread right, I'm having the same problem I can assign button and toggle mouse on/off - but while in mouse mode, the D PAD Center/OK button doesn't have any apparent effect. (Launching an app, etc.)

[virresh]

Hi @CrueRock , Android 12 seems to be affected by a different issue. See https://github.com/virresh/matvt/issues/55#issuecomment-1328190327

I have identified the root cause and will send out a fix for testing. Let's continue Android 12 related discussions on that thread and leave this for Android 11.