search

virresh / matvt

Virtual Mouse for Android TV that can be controlled via remote itself.
GNU General Public License v3.0
211 stars 35 forks source link

MATVT v1.0.3 issues #8

Closed lmpina closed 2 years ago

lmpina commented 3 years ago

I've posted an unlisted video titled MATVT v1.0.3 issues at https://youtu.be/aXtHGwBvb7w It's not searchable. Please share the URL as you see fit.

virresh commented 3 years ago

Thanks @lmpina I really appreciate the time you took to elaborate on the issue you're facing on your device. The URL will stay here as long as you don't redact it. I've posted an unlisted video from my perspective and I hope it clears some understanding of MATVT functionality. It's running on MATVT v1.0.3. Please find the video at https://youtu.be/sIkqsACMM5s

I hope our video documentation will help some future community contributor who has a TV Box setup similar to yours and are able to troubleshoot the issue and come up with the solution.

Again, extremely grateful to you and other generous folks who help improve this software.

lmpina commented 3 years ago

I believe I know why it's been so difficult to duplicate these problems. Android TV boxes invariably have mouse drivers in firmware. The drivers vary from box to box (and from firmware update to firmware update) because because the remote controls supplied with the boxes are all different. In Mecool's case, the cheapest boxes came with remotes having a hardware mouse toggle, but no voice assist/microphone. The more expensive boxes came with remotes having a voice assist/microphone, but no hardware mouse toggle. Mecools even have built-in support for the FireTV Stick remote. Surprising as it seems, you can easily pair a FireTV stick remote to a Mecool. There are also various drivers in firmware for game consoles. These drivers are conflicting with MATVT. Unless these same popular remote configs are installed on the development system, I can't ever imagine your going to be able to duplicate the otherwise easily verified problems, which vary from box to box, from firmware to firmware, etc. That's my thinking right now. I could, of course, be wrong.

lmpina commented 3 years ago

I've posted a second unlisted video titled "Variables Affecting Pointer Accuracy" at https://youtu.be/7TZS9WQtJxw Same as before, it's not searchable. Please share the URL as you see fit.

virresh commented 3 years ago

Thanks a lot! The experiments in the second video does explain some improvements that I can do. ----- In case interested in the technical part ---- I'd added legacy behavior (which works without sending touch input and if it fails, then it sends the touch input as fallback) to support some issues with button clicks not registering (especially on TV home screens, because most TV manufacturers don't support Touch input on their home screens, because they are not expecting anyone to be able to send touch input.) ----- End of technical part ----

However, I believe that even though the pointer is moved downwards on the next render, click location when the original button was pressed should not be different (another click of the dpad center button would be at a different location however).

Also thanks for showing how to enable the status bar. I'll try it on the emulator if possible and see if it'd be possible to get the mouse over status bar

It is indeed possible that there are some driver specific issues, however I've actually tested it with my own Android Smart TV (Sony Bravia kd-49x7500h) and use MATVT on a daily basis, so I'm pretty sure that the emulator is equipped with common drivers. I borrowed a firestick remote from a neighbor and that also works just fine. The issue is most likely the way I'm sending in touch inputs (which cannot be changed because I don't ship hardware with MATVT due to which the TV will never recognize MATVT cursor as a real mouse pointer).

I think I've some ideas on improving the app. If possible, could you report what happens when:

My apologies if I'm asking too much information. Thanks a lot for the experiments! I have some pointers to work on. Mouse jumping around is definitely not an expected behaviour. I'll get to it over weekend 😄

lmpina commented 3 years ago

If possible, could you report what happens when:

• GBoard is installed from Aptoide Store? (You can rollback to TV version by uninstalling updates and then reinstall latest version from playstore)

As far as I know there are two, possibly three very different apps called Gboard.

  1. (White-color) Gboard - the Google Keyboard - Minimum version Android 6.0+
  2. (Black-color) Gboard - the Google Keyboard - Minimum version Android 8.1+
  3. (?-color) Gboard - the Google Keyboard (Android TV) - no experience with this one...

Now, here's where it gets interesting:

D-Pad mode can NOT enter the keys or click the microphone button on the white keyboard. Only mouse mode can do that. This is true for both hardware and software mouse toggle.

The black keyboard works exactly the opposite way!

Mouse mode can NOT enter the keys or click the microphone button on the black keyboard. Only D-Pad mode can do that. Again, this is true for both hardware and software mouse toggle.

It doesn't matter which version of the white or black keyboard is installed, and it doesn't matter where you get it from (Aptoide, APKPure, APKMirror, Playstore, etc.). The result is always the same. You need a mouse pointer to use the white one. You need a D-Pad to use the black one. The black one is definitely preferable since it allows the microphone on a Mecool ATV3 IR/BT Remote to voice search within apps, but since the black one requires Android 8.1+, it doesn't help M8S Pro owners with Android 7.1 or 7.2. M8S Pro owners have to use the white Gboard and they need either an aftermarket remote with hardware mouse toggle or the bundled RemoterATV3 with mouse-toggle software in order to click that all-important microphone button.

Now, regarding the white keyboard, MATVT accuracy is way OFF on M8SPro series TV boxes with Android 7.x, but spot ON (as far as I remember) on the Tanix TX6-p with Android 9. The Fluxii app couldn't click anything with accuracy on the TX6-p. 

• Is Fluxii mouse toggle able to move over the status bar? If so, I can be certain that it's possible to do with software without root privileges and try to investigate more on this.

Neither MATVT nor Fluxii could click the Status Bar on the Tanix TX6-p. MATVT's pointer slid underneath. Fluxii's pointer stopped at border. Fluxii's pointer could not enter it.

• Does the Fluxii mouse pointer also face issues with drawing over overlays? If not, I believe there's some hope to improve this behavior as well.

It varies. When testing on the H96Max RK3318 TV Box with Android 9, neither Fluxii v1.05, nor MATVT v1.03 could click the activation buttons of either of the two screen recorders installed.

But when testing on the Tanix TX6-p TV box, also with Android 9, MATVT had no problem clicking the activation buttons with perfect pointer accuracy. I don't remember whether Fluxii could click also them on that box. Fluxii's pointer accuracy was so far OFF that it was very frustrating to use (same frustrating experience as when using MATVT on Mecool boxes with Android 7.x).

• Do you also face issues in clicking on overlays on any of your devices? In the first video I couldn't understand if the overlay clicks were not registering.

That's because of the pointer accuracy problem occurring on the Mecool M8S Pro. In that video I would be clicking (repeatedly) right on top of a key located on the upper row of the keyboard but instead of registering on the key, the clicks registered 3/4" up and to the left, into the black area above the keyboard - so nothing happened. Might have been more obvious if I'd thought to turn on pointer location in developer options...

virresh commented 3 years ago

Thanks @lmpina This feedback is very insightful

For the black vs white gboard versions, I believe that one of these has touch input enabled, the other doesn't. The one with touch enabled would work fine with MATVT, the other one would won't at all. Can confirm that white gboard doesn't work with DPAD mode though :(

I believe the issue is with how IME is ignoring any touch input we try to give them. (Which is why even hardware mouse failed). I'll try to see if there's another way around it.

As for pointer accuracy, I have some legacy behaviour that first focuses on the object and then clicks. I believe an option to disable that should solve the problem. I'll do a pre-release in a couple of days and ask you for feedback then

lmpina commented 3 years ago

Two requests for next development cycle:

  1. Please consider adding version number info to the MATVT configuration window. The upper left corner should indicate "MATVT - Mouse for Android TV Toggle v.1.0number."

  2. Please address inconsistent mouse pointer duration. Right now, MATVT's pointer might sustain 25 seconds or more where it's least needed (on top of a movie) and only 2 seconds or so where it's most needed (on top a keyboard.) Combine short pointer duration with pointer inaccuracy, and it's twice as hard to hit the key you're aiming for. When you have to click repeatedly to bring out the pointer, you invariably click on yet another wrong key. For what it's worth, remotes with hardware mouse toggle keep their pointer's out for a consistent time. It might be as short as 6 seconds. It might be as long as 15 seconds. But, whatever it is, it's consistent. It doesn't change depending on what task the mouse is attempting. If it were my decision, I'd split the difference and go for something like 10 seconds. Just fixing this would eliminate 50% of the frustration currently experienced when using MATVT on Mecool M8SPro TV boxes with Android 7.x

Thank you!

lmpina commented 3 years ago

In the spirit of hopefully helpful feedback, I installed MATVT v1.03 on a Mecool KM9 Pro Android 10 TV box with the following results:

  1. Pointer accuracy is not spot on, like it was on the Tanix TX6-p Android 9 TV box shown in the Factors Affecting Pointer Accuracy video, but it's so close to spot on, that the tiny error observed does not cause any problem. Interesting that MATVT's pointer accuracy can vary from spot on, to just slightly off, to way off, depending on the TV box. The same happens with Fluxii mouse toggle for Android TV. Yet, Fluxii mouse toggles for FireTV (two apps - one for 1080p and one for 4K models) both work perfectly (on FireTV products). I wonder what the difference is...

  2. For use with Google Chrome on Mecool boxes with BT remotes where the MATVT boss key is 4, the Return key (only option), it would be great to have an adjustable key-press, latency setting. Right now, MATVT v1.03 enters scroll mode too quickly on the KM9. When on the Home page, Chrome uses Return to popup a History menu. This interferes with MATVT mode switching. Other times, you just want a normal return when using Chrome, not entry into scroll mode. Both problems are easily experienced. Just set the boss key to 4 and see what happens in Chrome. I'm thinking if MATVT's entry to Scroll mode could be delayed just a little, these problems (with Chrome) might be solved.

Thank you!

virresh commented 3 years ago

Thanks @lmpina A bit bogged with other work, so couldn't look into it this weekend

Please consider adding version number info

Will surely add this somewhere on the configuration window

Please address inconsistent mouse pointer duration

Noted. I'll try to address this. It is doable, but might take a couple more weeks 😅

pointer accuracy can vary from spot on, to just slightly off, to way off,

Can't comment. Maybe one day if I get my hands on a machine which has this behaviour, I'll be able to better analyse this. But thanks a lot for your experiments and insights. Definitely helps to have some experiments in case some other contributor decides to work on accuracy 😄

entry to Scroll mode could be delayed just a little

Sure. I think I can add a delay and make it configurable. Will try to get this in the next release also

Thanks again!

ghost commented 3 years ago

Y

Lll

lmpina commented 3 years ago

Virresh,

Time to close this pull request? I read about the difficulty you're having with others trying to steal your software. I know that you're discouraged by that and not inclined to continue development of MATVT. I have no idea what this person is trying to say. Although there's been a surprising number of views of the two (MATVT developer) videos I put up on YouTube (indicating that all of those people visited your Github site), I'm thinking I should take them down and apologize for having bothered you.

Best regards,

Lawrence

On 5/26/2021 11:56 PM, gvejar85 wrote:

Y

Lll

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/virresh/matvt/issues/8#issuecomment-849299328, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATEAWRQ7WULQFQAMAHZMAC3TPW7FDANCNFSM4ZVSJXEA.

virresh commented 3 years ago

Hey Impina, I appreciate your concern but I would like to express my gratitude to you for pointing out flaws and then helping with experimentation. Your videos are a valuable resource for any potential programmers who would be interested in contributing to this project.

As for me, I am lately not getting enough time to work on this project, however I haven't yet lost all hope and will be back with improvements sometime. It's just that I have some other free and open source projects as well, and I find it hard to distribute whatever little time I have after my day job. The fact that people think it's okay to sell off an open source project just because it's free doesn't help either.

On the brighter side, the feud over the person selling my project was painful, but is resolved now. I'll circle back to this issue sometime in future and try to resolve it. And rest assured, I am not troubled with any constructive feedback / criticism for the project. I welcome it whole-heartedly 😄 . Though it does get weird sometimes with comments such as the one on this thread. Apparently the person who posted the comment has deleted their account so we can't get any information on what they were trying to say.

virresh commented 3 years ago

Hi Impina, While there are a lot of issues here, I have only been able to address one

inconsistent mouse pointer duration

This should be consistent now (and set to 10s, not configurable currently). There are other things that have been addressed though, do try the v1.04 release. I don't think it would help with pointer accuracy, but might make a difference with some apps that didn't respect MATVT cursor input before.

Link: https://github.com/virresh/matvt/releases/tag/v1.0.4

virresh commented 2 years ago

Closing this issue for now. Don't know if anything can be done about pointer accuracies, other items should be incorporated in previous releases.

Please feel free to re-open or open another one if needed.