Why the old relative tap coordinates (in %) were switched over to absolute ones?

[esauvisky]

Hey there! I'm the forker from Reditt :grin:

So, just tried the script "for the first time" (well, the new version at least), and unfortunately it doesn't work out-of-the-box as it used to, because the X-Y coords on the YAML are no longer relative – i.e.: in percentage values – like they were before, and the default ones, even though my screen is 1080p as well, are off by 20-30px each.

Did this change for any particular reason? Otherwise, I'd be interested in bringing back the feature, as it's a deal breaker entry barrier for non-technical users.

Gimme your thoughts!

[Azelphur]

It changed because percentages don't work across different aspect ratios, so it would not work on newer phones with 16:10 displays or tablets at 4:3, etc

[Azelphur]

Also, some 16:9 phones have soft buttons, which further muddies the waters.

I've been thinking about adding profiles, something along the lines of 1080p no soft buttons profile, and then a list of phone models that profile would work with, so that profile is automatically loaded if your phone is in the supported list for that profile.

[esauvisky]

It changed because percentages don't work across different aspect ratios, so it would not work on newer phones with 16:10 displays or tablets at 4:3, etc

Yeah, but most phones (~80%) are 16:9 these days, and with absolute coordinates relying on actual number of pixels, as there are many 16:9 resolutions, the overall percentage of devices that would work out-of-the-box is much smaller (take my device, for example, even though it's the same 1080p it doesn't work).

Also, some 16:9 phones have soft buttons, which further muddies the waters.

I've been thinking about adding profiles, something along the lines of 1080p no soft buttons profile, and then a list of phone models that profile would work with, so that profile is automatically loaded if your phone is in the supported list for that profile.

The old code figured out the soft buttons bar (quite an elegant solution if I'd say), and it would set the device resolution to the actual resolution minus the bar's height. Do you think this wasn't robust enough?

[esauvisky]

I think a great implementation would be to use relative coordinates expressed in percentage (along with the softbar check, of course) for every X/Y position that's empty or not defined in the YAML file. Otherwise, it sends absolute X/Y taps defined in the config normally.

That way, we could have a default config.yaml (with empty default values) that would work on most of the phones out-of-the-box, and corrections could be made if need be.

To me, this looks like it would bring the best of both implementations together:

• The user would not have to calculate the relative % values from the absolute ones.
• And he would be able to fix a single step, e.g.: if the script misses a single tap in the middle of the process.

Can I give it a try or do you think it's not worth it?

[Azelphur]

Actually, most (new) phones are not 16:9, going through a list of the current flagship phones:

Galaxy S9(+): 18.5:9 Galaxy note 9: 18.5:9 Pixel 2 XL: 18:9 Pixel 2: 16:9 OnePlus 6: 19:9 LG G7 ThinQ: 19.5:9 Huawei P20 Pro: 18.7:9 ...

You get the idea. We definitely need to support variable aspect ratios, it's not optional anymore.

The default settings are geared for a 16:9 1080p phone - and I think most phones that are 16:9 are also 1080p? Does your phone have a soft nav bar? that could be pushing the numbers off?

[Azelphur]

As for soft bar detection, yes that worked reasonably well (and arguably I could do a better job by inspecting build.prop) but like I say, we must accommodate variable aspect ratios with whatever approach we take.

This is why I like the profiles idea, the only other option is some sort of algorithm that figures out where the buttons would be based on the phones aspect ratio, which I'm sure is also possible.

[esauvisky]

The default settings are geared for a 16:9 1080p phone - and I think most phones that are 16:9 are also 1080p? Does your phone have a soft nav bar? that could be pushing the numbers off?

Yes, it does, and it's definitely not the standard one :grin: I've got a Google Pixel with DU on 8.1 and a weird navbar called bling or something like that.

---

Yeah, I get the point, unfortunately there's not much we can do to automate that step of the process. Well, the least I will do then is work on documentation. I'll create a # Installation/Configuration section on the README, as there isn't even a mention of the clipper service there :grimacing: Also, I've got the best QA I could have for that: if my girlfriend can make it by herself, it means it's good enough for most users :grinning:

Expect a pull request soon!

[Azelphur]

Thanks, docs definitely need work, appreciate the help on that!

Another idea I had, on top of profiles, perhaps a configuration wizard could make it easier, essentially: Grab screenshot, open it, ask user to click on the screenshot where the rename button is, then where the favorite button is, etc etc.

[esauvisky]

Yes, I thought of that as well followed by "hmm, the hassle of implementing an UI, only for fetching tap positions once per user :thinking:" I wonder if it's really worth the trouble tbh.

I never played much with GUIs on python, though, so if you happen to have anything in mind already, any libraries that could be a good candidate for developing a solution like that feel free to give me a shout, I'd certainly enjoy giving it a shot :smile:

Useless information: I used to do something similar and ~quite~ very sketchy in an old PoGo renamer of mine — a bash script that used GoIV, so yeah... There's an application called scrcpy that not only mirrors your device screen via adb as also lets you control it with your mouse as well, with ridiculously low latency. Quite an awesome piece of software if I say so, take a look at it if you don't happen to know it. The way I did it was by telling the user "Place your mouse on top of button A and then press Enter" and then I'd fetch scrcpy's window position and calculate the relative position of the mouse against it.

[esauvisky]

Oh, forgot to mention, one thing the script could do to help the user get the coordinates is programmatically enable Pointer Locations via adb, quite nifty:

# Android 7 and higher:
adb shell content insert --uri content://settings/system --bind name:s:pointer_location --bind value:i:1
# Android 6 or lower:
adb shell settings put system pointer_location 1

This might come handy if we go the wizard route as well ;)

[esauvisky]

Ok, nevermind everything above. Tap a spot on your phone and run this. Seriously, do it:

adb shell dumpsys input | grep RecentQueue -A10

:grinning:

[Azelphur]

That's a very cool find. I assume you're thinking that would be good for creating a setup wizard? :)

[esauvisky]

Ohh, you betcha! :grin: I'm gonna give it a go.

Though first I gotta get the script actually working, something funky's going on with the IVs regexs :confused: Will open a new issue.

[Azelphur]

Sounds good, I will leave that to you then!

I would suggest making it a separate python script (which would be included in this repository of course), and using an approach something like this: print("Go to your pokebox, select a pokemon") input("Press the rename button, then press enter") input("Press the ok button on the on-screen keyboard, then press enter") input("Press the ok button to rename, then press enter") ...

Should work well - open to suggestions of course :)

[esauvisky]

Yup, definitely going that route... no way I'm messing around with buffering an input device's dump :joy:

That will certainly suffice as well, is not that big of a deal anyway. I'm actually working on something else that's a bit more critical, pull request coming in a moment.