Edge case: During result screen, Servant Kizuna Levelup / New CE Gain will cause the script to stop

[raspberrytealeaf]

I'm not sure how to describe this behavior, cause I've never seen it happen in person. I always discover it when I come back [x] hours later.

I upgraded to the latest version of Nox recently cause I heard the date issue is fixed. But after upgrading, I can't seem to leave the bot running for multiple hours at a time without the script breaking or pausing. The most common place it seems to "pause" is on the drops/exp/etc screens after battles. I'll come back and it's on the drops screen.

If I click the next button for it, it'll continue on without any issues, but I don't know what happened that made it get stuck.

[ParasiteEve]

Happens to me, I have to check every hir

[ryuga93]

https://github.com/29988122/Fate-Grand-Order_Lua/issues/109 Side notes: I have no idea it's because I updated Nox to 6.2.5.3, or I updated Ankulua to 8.0.5. The performance of recognition is VERY FAST now. You guys gotta give it a try.

There might be performance difference with the newest version for either Nox or Ankulua. Try to update to newest Ankulua.

[raspberrytealeaf]

I'm using both Nox 6.2.6.0 and Ankulua 8.0.5 already. :(

[TheIanator]

I think when two or move servant get bond up at the same time, or when you get a master/mystic code level it takes an additional click to get though that isn't accounted for in the script, could this be it?

[raspberrytealeaf]

Afaik, the script already accounts for that. I've gotten servant bond level rewards (aside from CEs for lvl 10) w/o having this issue.

[potchy]

We'll probably need a video to understand what's going on. To be honest, it happened to me once or twice during the event, but I didn't give it much importance.

[TheIanator]

Both videos on latest master version.

https://gfycat.com/VelvetyHandyElver

Script stalls at item drop screen after taking an additional click to advance through a mystic code level up.

https://gfycat.com/GlossyAcrobaticCockerspaniel

Script stalls at Bond Points screen, not what I expected. Expected it to take an additional click to advance through two bond levels and get stuck at Items Dropped like the other, instead never clicks at all. Perhaps Lancelot's head blocking where result.png is looked for is breaking it?

[29988122]

IRRC, I did add an edge case check for servant Kizuna LVL up/Mystic Code LVL up

Maybe

potchy

should take a look at it

or

We just wait for our debug logging to be completed..

[TryBane]

We have a case that checks for bond level up, but only 1 in any instance. And not when Mystic Code levels up. I implemented a different method during this particular section where it just continuously clicks about 12 times, which is enough to get through all of that assuming 5 bond ups and a mystic code up, then waits to see if friend request screen comes up.

I may also need to implement a case scenario where you might get a bond CE as well, since that screen is also a bit different.

[potchy]

I second @TryBane's idea. Clicking the screen X number of times after result.png shows up is good enough, faster and just easier to implement, honestly. The client doesn't communicate with the server during this time, so it should work for all regions, even if the connection is lost somehow.

*waiting for a pull request*

[potchy]

Hey, I managed to get screenshots from the Bond CE screen. Unfortunately, the video recording got screwed up. Only the first frame got recorded. Thanks a lot, NVIDIA!

[TryBane]

That should be enough I think. The clicking for me is a long enough time to click through the first section, then we just need to image search for clicking "close". I believe we should search for the star image, no writing, to make it work on all versions not just EN. We can have a slight increase in region we are searching in so that if things get added it won't cause issues. Then just click the top left.

[LambdaTenEleven]

Same problem here. I have to skip results screen every time. After battle script just stalls at results screen.

[TryBane]

Is that using the most recent version of the script? I believe I have solved a normal bond level up and mystic code level up. The only currently unhandled case is when you get a max bond CE.

[LambdaTenEleven]

Yes, yesterday I downloaded the most recent version. Everything worked fine except skipping the results screen and support selection. When battle ended the script just stuck at results screen. Bond lvl up screen or something like that hasn't appeared. So I skipped it by myself and it started the quest again. When it clicked the quest button it stuck again, however I set support selection to "first". Today I re-downloaded the script and now everything don't work except for quest selection function :/ Sorry for my English btw

[TryBane]

I'll do a quick modification on the support script but it should still work. I just redownloaded and tested it and it worked fine for me. Not sure what is going on. Are you on a device that uses a different aspect ratio than 16:9? For instance, do you have bars on the left/right or bottom/top?

[TryBane]

Nevermind, I didn't make a change because it should still work without issue selecting a support. Especially if it is to select first. Can I see your configuration if you can get a screenshot?

[LambdaTenEleven]

My phone is Xiaomi Mi A2 Lite with Android 9 (Android One official firmware). Screen parameters: 5.84", 19:9 and 1080x2280 pixels. Also, it has a notch (can it be the reason?). I've tried to launch it without any editing. SupportSelectionMode set to "first" by default. Nothing happened after hitting the OK button. Here is what's my screen looks like (script isn't launched): Then I decided to change screen resolution via ADB to 1080x1920 (16:9). After that all next screenshots have been shifted and clipped. However, while playing everything looked fine. It might be an Android bug. The script hit the quest button and froze. With 720x1280 resolution it's looks more awful and quest selection don't work. Before last update it worked only with 1080x1920. As I said earlier everything worked fine except skipping the results screen and support selection. So the problem is 19:9 screen?

[TryBane]

I can tell you right now that it is PRECISELY why. The aspect ratio of your screen is so important because it determines the locations of the clicks, where it looks for images, and uses it to scale the image it sees and tests it against the image saved. If your resolution is even slightly off, it won't work.

Give me 2 minutes here I have to make one adjustment but you can give my own branch a try. I haven't been able to test to ensure there are no bugs but I have a friend who has been able to use it with different aspect ratios and it works.

Here

[LambdaTenEleven]

I've tried your version but, unfortunately, there is an error :(

[TryBane]

Sorry, I'm an idiot. Give it another download XD I made the xOffset and yOffset values local and didn't leave them as globals. Sorry!

[TryBane]

These small bugs are why I need people to test :P

[LambdaTenEleven]

Now there is another error:

[TryBane]

That's an easy fix, one second and I'll update this message when it's fixed...

Edit: Fixed

Edit 2: I appreciate your patience :D

[LambdaTenEleven]

No problems :D So something went wrong again:

[TryBane]

Sorry, didn't get a notification. I fixed it.

[LambdaTenEleven]

There is no errors now but it still doesn't work for me :( After clicking ok nothing happened. I changed resolution via adb again and it still only clicks quest button. I think my phone is the main problem so I'm going to test your script on my 1280x800 tablet tomorrow. I'll reply if it works or not. Thanks for your help. Hope my English is good enough :d

[LambdaTenEleven]

Hi! I've tried your script on my Asus Z380M tablet (1280x800, 16:10 or 8:5). So it has the same problem as my phone. However, while testing I've noticed this tiny red rectangle (sorry for photo, my volume keys are slightly broken): It appeared for one second and then gone.

[vit2657]

I also gave a try to your modified version with screen resolution on my poco f1 @TryBane (which has weird resolution of 1080 x 2246). Currently having same problem as the comment above. Although I'm mainly using this script on Nox, I will be happy to help you test your modifications with my device if needed (I'm only using it for development anyway).

[TryBane]

For the both of you, please go into the FGO_EN_REGULAR file and scroll to the bottom. You will see a variable there called DebugMode which is set to false. Please change that to true before continuing. I would like to get some of the information that will pop up on your screens to help solve this issue.

Firstly, there will be a notification when you first start the program providing the resolution the game is reading. I'd like you to get that for me and post a screenshot.

Next, every time at the main menu, it will provide the offsets that I've calculated using the screen resolution, so if you could get a screenshot of that, I'd appreciate it.

After that, there should be red boxes popping up in the support screen like above, and a few more. They should be popping up in about 4 places total if I recall correctly.

If you could get screenshots of those then I'd like to work on getting on calculating what the issue is.

[TryBane]

As an update, I've gotten a possible solution implemented in the current build located in the same place. Give it a try and tell me if anything doesn't work appropriately. Should that be the case, please turn DebugMode to true and return with screenshots so I can further investigate what might be causing the problem.

[LambdaTenEleven]

Still have the same problem :( Some info from debug mode: If launched from battle screen Resolution: X: 2043 Y: 1080 (Real resolution is 2280x1080) Offset and first red box: X: 240 Y: 0 Second box: Third: The last one: Main menu First red box after clicking OK The only red box in the support menu:

[TryBane]

Alrighty and this is with the more recent change I've made? If so, this is going to be a bit annoying.

[TryBane]

Okay, I am not entirely sure what is going on in your case as far as how your phone decides how to position the game but your phone is dumb. However, I do at least have a solution specifically for you that you can use. All you need to do is make xOffset equal to 300. Right now it gets set to 240 because the instances I've run across where the resolution isn't 16:9, it takes the extra space and simply cuts it in half dividing that extra space between both sides of the screen.

And so I calculate that total space, then divide it by 2 to get the offset I need. In your case however, the total extra space is 480. Instead of dividing this in half and splitting that onto both sides, it gives one side 300 pixels of bar and the other gets 180. I haven't a clue as to why.

If you make the offset what i just said, it should be fine. To do this, open your FGO_EN_REGULAR file from my repo and at the top you can delete the entirety of calculation I do when setting to xOffset (do NOT delete the script and compare that is directly under it) and replace that whole calculation with 300. I will continue to look for a solution, but it just seems dumb. Rather than have you constantly trying to figure it out I just figure give you the softfix for now.

This is a solution for anyone who doesn't have a 16:9 resolution but rather than give each individual an offset to use specifically for them, I wanted to make an algorithm that can do that automatically. Sadly, instances like these exist. I'll have to try figuring out another solution or improve the one I've got to consider non-even distribution of bars.

[vit2657]

Finally got a hand on it too. In my case it is calculating x offset of 217.7~ which is incorrect. Hard-coding 280 does fix it however. Problem in my case is a space where notch is. Game does not render there, even though it does render itself on the right side (blank space).

Edit: 280 was out of the ass number by me, which just works, probably should re-calculate it like you did for the guy above though.

[TryBane]

All I did to see what the ACTUAL numbers were on each side was bring a single image he posted into this image editing site, and resized it to 720 on the height (keep aspect ratio option turned on). Make sure you hit "Apply" then choose the Crop option. A red box will pop up, which you can click and drag the center of to move, and clicking slightly inside the box at one of the edges will move that particular edge/corner so you can resize it. I dragged it to the left, and resized it till it was approximately the right size of the left part of the screen. Look at the Width of the crop box on the left side and double that number.

That is all you need to do to determine the exact offset value you need (you may be able to use an easier method than this but this was the one I used). It is important to note that it is the LEFT side you need to measure, not the right. And if someone's aspect ratio results in a set of bars on the top and bottom, it is the TOP that needs to be measured in those cases.

[TryBane]

Which btw, your eyeballing is REALLY accurate. The measure I got from resizing your screenshot to 720 was 137, which doubled is 274 (for those that don't want to math). So being only 6 pixels off is really impressive :D

[TryBane]

One thing I realize for an instance like how you have a notch on your device, is that if you flip the device over it might suddenly not work. I don't have an idea as of yet on how to fix this potential issue, so do me a favor and check if that DOES cause an issue, and let me know whether it does or not cause whatever solution I may make regarding your guys' cases will NEED that info.

[vit2657]

Yeah, it does not work if I flip it, which is logically correct. It just constantly spams a toast with info about xOffset and yOffset (280 and 0 for now) trying to re-determine it probably.

One thing to note is that a black bar on the side with the smallest offset is a safe space so the game won't render there cause of rounding corners of the screen.

[vit2657]

And when I removed the hard coded value, the offset initially calculated appeared to be rather big (217 currently).

[TryBane]

Yeah, that is because of the lack of equal distribution of the bars. I think I may have thought of the concept of a solution for all of this, but actually coding it might take a bit. Potchy said before that resolution algorithms are his thing, and after he gave the go ahead I started to work on this particular iteration myself. Maybe he might have an answer that I don't have, but I will be looking into seeing about getting my own solution to work appropriately. For now, I suggest keeping the device flipped as it was, or modifying the script in order to handle the opposite orientation. Whichever is most comfortable for you.

I may ask for your help about testing a later version when I get around to wrapping my head around the idea that I have and actually implementing the first version of it. Thanks for your patience and feedback about the issue you were having. It has helped me understand some of the problems that trying to take care of screen resolution comes with :D

[LambdaTenEleven]

@TryBane Thank you for your effort! Now everything works perfectly!

[vit2657]

@TryBane Yeah sure, let me know when to test it more. I saw in another thread that preferred support selection is not working right now, so I cannot fully use script on the phone though. Mainly on the Nox for now. But will switch for tests if needed.

[TryBane]

Well specifically support selection doesn't work when looking for CEs in EN server due to the % bonus display slightly overlapping the CE image. It will still work in the case where you are only looking for Servants, and you can probably recrop the image (or make a new one) that will fit.

IF you are referring to the other thread where I was helping another person with Android 8.0+ and that had an irregular aspect ratio (was helping him at the same time as you guys) then the band-aid fix for that is to reduce swipes to 0 and increase refresh times as appropriate. It isn't an ideal solution, but it works until an ideal one can be thought up.

[vit2657]

Yeah, was referring to the second case (low-key fixed first one by re-cropping image on the Nox, was an easy solution though). Saw the temp solution too of cause, there is just no need for me to use a phone specifically (though it would be nice in a future to move to it completely), so will stick to Nox for now.

[TryBane]

Alright, that sounds good. Did you specifically run into the issue? Or just noticed someone had one? It isn't related to the resolution issue if that is a thought. But regardless, if it works on Nox and you don't have an issue running it there, then sounds good to me! I'll still see about getting this solved for those would-be phone users of the script when there is a bloody notch in their device (which I may be getting one with the S10+ >.> )

[29988122]

Tested myself, servant Kizuna level up screen can be solved by the new ContinuousClick(). That leaves us CE gain screen.

Can anybody tell me the sequence of the whole result screen(edge case included?) Gotta document that in case DelightWorks made any change to it.

What I know now(could be incorrect):

Servant Kizuna increasing animation
Servant Kizuna level up (OPTIONAL)
Master and Cloth Exp
Reward List
Event Reward List
~~~~~~~~Long Loading~~~~~~~~~
Stage Reward(Quartz, Mana Prism, etc)

Where is CE gain located?

[ryuga93]

https://youtu.be/Ek3Se0sAi8c?t=128

After Kizuna Level Up

[29988122]

Ahh, I see. No "favorite tagging system" available in the US server then.

Guess I'll use the border of the "status" button as recognition region then.

1. It's always red(selected), and distinctive
2. The location always stays the same
3. The area's large enough, and we can avoid language-dependent text. My brain farted.

---

So is this correct now? Added a few more entries

Servant Kizuna increasing animation
Servant Kizuna level up (OPTIONAL)
CE GET, Click anywhere to continue(OPTIONAL)
CE detail screen, click on the top-right corner to continue(OPTIONAL)
Master and Cloth Exp
Reward List
Event Reward List
Send Friend Request or not dialogue(OPTIONAL), click certain button to continue
~~~~~~~~Long Loading~~~~~~~~~
Stage Reward(Quartz, Mana Prism, etc), click anywhere to continue

[TryBane]

You actually could use the favorite button. It is in the EN server since the Thanksgiving update I believe.