steve1316 / granblue-automation-android

Educational application written in Kotlin aimed at automating user-defined workflows for the mobile game, "Granblue Fantasy", using MediaProjection, AccessibilityService, and OpenCV.
64 stars 11 forks source link

android version not working #6

Closed masayuki232 closed 3 years ago

masayuki232 commented 3 years ago

Screenshot_2021-09-04-06-26-20-83_6948ba2d6cf6960c4526b24e4863ae4e the bot cant seems to work on my android device. using oppo a5 2020 running android 10 . tested on skyleap, chrome and apps. says it cant find the home button.

steve1316 commented 3 years ago

It seems that your device is 720 pixels in width. I will get to work supporting that screen width as soon as I have some free time.

steve1316 commented 3 years ago

Just updated the bot to v2.5.1 and added preliminary support for devices like yours with 720p in width.

In addition, I included a new setting that will allow you to set your own image scale. You can find out your scale by taking a screenshot directly on your device and then pick a easily recognizable image asset from the /images/folder in this repository and scaling it until it relatively matches the size of the asset in your screenshot. Whatever percentage you scaled the image asset by will be the number that will go into this new setting.

Let me know how it goes!

masayuki232 commented 3 years ago

Screenshot_2021-09-06-14-53-43-32_6948ba2d6cf6960c4526b24e4863ae4e

just tried and got this error.

steve1316 commented 3 years ago

Hmm, so the scales of 0.65, 0.67 and 0.69 does not seem to work for your 720p screen. Can you go to the Settings page and set your custom scale starting from 0.50 and working your way up to something like 0.95 going 0.02 at a time and repeatedly running the bot until it actually detects the HOME button? After that, post the scale that worked for you here and I will update the bot to create a range from your scale.

EDIT: Initially when I created the scales of 0.65, 0.67 and 0.69 for 720p, I resized my own device's screenshot from 1600x2560 to 720x1080 and then resized my image asset of the Attack button by 67% to match what was on my screenshot.

masayuki232 commented 3 years ago

sure , will update later

masayuki232 commented 3 years ago

Screenshot_2021-09-06-16-27-44-17_6948ba2d6cf6960c4526b24e4863ae4e

fyi, the error shows up different when i set up ths custom scale. perhaps by chance the 1st error isnt related to the screen scale?

steve1316 commented 3 years ago

What dooms it all is that it failed to find the HOME button in the first place and a cascade of failures led it to stop there when it attempted to open the Menu to find the Event banner. That cascade is caused by the fact that the bot cannot find match any image assets at all on your screen using the current scale(s).

I thought I had the solution by just simply resizing a screenshot from my device to match your screen dimensions and obtaining the scales from there, but I'm ultimately not sure. But I do know the method works as that was what I did to support my Tab S7 tablet.

EDIT: I will see if I can come up with a better solution tomorrow.

masayuki232 commented 3 years ago

thanks for the hard work. for more information, i also tried setting custom scale as instructed, from 0.5 to 1.0 going 0.02 each time. still have no luck

masayuki232 commented 3 years ago

Screenshot_2021-09-06-16-56-53-51_6948ba2d6cf6960c4526b24e4863ae4e

setting to 0.67 makes the bot confirm the location at home but cant find home menu button

steve1316 commented 3 years ago

Oh thats weird. Can you set the confidence of single match and multi match lower than their default 80%? Maybe 75% or 70%?

masayuki232 commented 3 years ago

Screenshot_2021-09-06-16-59-08-53 this is my home screen

masayuki232 commented 3 years ago

setting confidence from 80 to 70 have no result.

kevin01523 commented 3 years ago

Hmm, so the scales of 0.65, 0.67 and 0.69 does not seem to work for your 720p screen. Can you go to the Settings page and set your custom scale starting from 0.50 and working your way up to something like 0.95 going 0.02 at a time and repeatedly running the bot until it actually detects the HOME button? After that, post the scale that worked for you here and I will update the bot to create a range from your scale.

EDIT: Initially when I created the scales of 0.65, 0.67 and 0.69 for 720p, I resized my own device's screenshot from 1600x2560 to 720x1080 and then resized my image asset of the Attack button by 67% to match what was on my screenshot.

finding the scale kinda hard to figure out

just asking what app etc you are using to resize compare the images at.

steve1316 commented 3 years ago

Just updated the bot to v2.5.2 with more detailed debugging messages and a new setting called Enable Test for Home Screen under the Misc section of the Settings that will test your device compatibility and if the current scale(s) failed, it will automatically find you a scale that works for you. @masayuki232 Let me know how this test goes for you!

@kevin01523 I took a screenshot from my phone and sent it to my computer. Then I opened IrfanView, a third-party alternative to the basic Windows Photo Viewer, and resized it by a certain percentage in Image->Resize/Resample. But now I have included a test that automatically finds the best scale for you, you no longer need to manually do this anymore.

masayuki232 commented 3 years ago

Screenshot_2021-09-07-10-52-51-02_6948ba2d6cf6960c4526b24e4863ae4e

Screenshot_2021-09-07-10-50-55-00_6948ba2d6cf6960c4526b24e4863ae4e the test worked but still got error when i set the custom scale to 0.65 as instructed

steve1316 commented 3 years ago

Can you take a screenshot of what happens after you tap the MENU button and post it here?

Like this: Screenshot_20210906-190608_Chrome

masayuki232 commented 3 years ago

Screenshot_2021-09-07-14-06-26-65_6948ba2d6cf6960c4526b24e4863ae4e it worked normally if i press the menu button then start the bot

steve1316 commented 3 years ago

Hmm, that is interesting. Is it consistently working after you yourself press the MENU button before starting the bot or is it also now working without you needing to do that?

EDIT: Also, I now have a discord server that you can find the link to at the top of the README if you need real time support or discussion. With GitHub, my response may not be instant.

masayuki232 commented 3 years ago

Screenshot_2021-09-07-15-40-36-38_6948ba2d6cf6960c4526b24e4863ae4e

kevin01523 commented 3 years ago

@masayuki232 try clearing data and reset the app after it crashes it might run after that i do experience this often too also move the play or the start button for the bot where it doesn't cover any buttons

steve1316 commented 3 years ago

User confirmed working after v2.5.3 update.