New JP UI Update - Not working with non-16:9 display

[buitim]

Before opening an issue:

• [x] I tested the latest release
• [x] I looked at other issues (even the closed ones)
• [x] I read the Troubleshooting Guide
• [x] I'll fill this template properly

Your issue will get closed/locked by the bot in case proper info isn't provided If you open unnecessary/dumb issues, you'll be rewarded with appropriate tags.

Describe the bug FGO JP just dropped an update where they made their UI adaptive. This removes the blue bars on the sides and messes up the areas that are checked. I personally think they're just out to get you lol /s

I haven't tested yet but it might still work on regular 16:9 screens.

Screenshots

Device Info (please fill at-least 'Device model'):

• Device model: Xiaomi Mi 9T
• Screen size: 1080x2340
• Android Version: Android 10 - Pixel Experience

[MathewSachin]

I personally think they're just out to get you lol

I don't think so. Full-screen display was something they should've done long ago.

But, I looked at a lot of screenshots and it just looks so unpolished and lazy. A lot of things are out-of-place. Guess they just released before properly testing out stuff. There is a work in progress mentioned there so hopefully they polish things up.

Even on 16:9 some buttons have moved up a bit, like the quest Repeat button. https://community.gamepress.gg/t/automatic-farming-app-fate-grand-automata/72155/2020?u=mathewsachin

I'd rather wait for them to get this messed up UI right, rather than adapt to it right now.

[reconman]

@MathewSachin Where did you find the screenshots?

[MathewSachin]

On Atlas Academy #general

[buitim]

I don't think so. Full-screen display was something they should've done long ago.

Haha yeah don't worry I was just joking around. Just some sarcasm :)

But, I looked at a lot of screenshots and it just looks so unpolished and lazy.

Agreed. It's nice to have but some elements definitely feel out of place. Feels like a hack job. I also heard the blue bars come back when using an NP.

[observeroftime02]

It still picks the correct battle node, that's about it, stops at the support screen.

Here is what it looks like on my Note 10+, yeah the border being gone absolutely messed this up, I am hoping for a fix before the next event, hopefully this isn't too hard to adapt to. If I can do anything to help just @ me.

[cherrysamasama]

It won't select the third skill of servants properly either.

[GilangAlRusliadi]

In my phone (Samsung A10S), automatic click button seem isn't in the right place (maybe it click on previous UI's place). So Arash didn't get full 100% for NP.

[VkIII3]

Maybe i'm saying a stupid thing (i have no idea how it works) but if we change the phone screen ration would it work? Anyway, thank you for everything you did...

[reconman]

@VkIII3 That requires a rooted phone and you can't just change it for one app.

Since only less than 0.01 % of Android users have rooted phones, that solution is not really usable.

[chairmanxyz]

Any chance at getting a temporary workaround for this before the JP Christmas lottery? I highly doubt we're going to see an official fix of the issues with this scaling anytime soon from JP, but the lack of this app will certainly make farming a lottery event much more tedious.

[sdarkpaladin]

Any chance at getting a temporary workaround for this before the JP Christmas lottery? I highly doubt we're going to see an official fix of the issues with this scaling anytime soon from JP, but the lack of this app will certainly make farming a lottery event much more tedious.

There are some that got it working in a PC emulator like bluestacks or Nox, so you could probably try those out.

Edit: I just tried it on Nox and it doesn't seem to work. Ran it on 16:9 and everything too.

[MathewSachin]

I've tried my best at it and I don't think support for wide-screen will happen anytime soon. It pretty much needs a rewrite of the app since the whole app's structure assumes 16:9 game size. Also, it doesn't help that FGO JP renders stuff mostly misaligned which means a single set of location coordinates won't work for every phone. Like if things were either right or left aligned, we could use same coordinates for different devices but they're doing it in some weird way right now.

I can make it work for a specific size like 16:9 or my phone's resolution but to support every screen size, the app needs to be pretty much rewritten which I don't know if it's the right time to do since DW IS going to change the UI more as they already said so in the in-game news.

For now, the best you can do is use a 16:9 phone or better yet emulator. You'll have to either adjust the Wave-number location or see #579 if you're using NPs every wave.

I'm also thinking of taking the app down from PlayStore for a while till wide-screen is supported. NA and other servers would eventually get the widescreen too, so it's not much use if we stay on PlayStore but don't support anything other than 16:9.

I don't mean that supporting all different resolutions is impossible. It is possible and I have tried that for some screens, but, it's more of a TIME investment than I'm willing to make on this.

---

Also, if you're not able to comment or open issues, it is because Interaction Limits are turned ON, which prevents new GitHub users from spamming. Did that because during Gilfest, there were a bunch of people spamming issues and not bothering to read Troubleshooting Guide or doing things I asked them to.

[potchy]

I'm fairly sure these options work even on unrooted phones: https://github.com/29988122/Fate-Grand-Order_Lua/issues/40#issuecomment-439693145 https://community.gamepress.gg/t/automatic-farming-app-fate-grand-automata/72155/2040

It might be annoying to have to change the resolution every time you wanna use the app, but hey, it's something! I used the first one back when we still didn't have the scaling algorithm figured out.

[thai-ng]

@MathewSachin I had a brief look at the code, and at least the skill coordinates are hardcoded. Then that sounds like we can have a "training" phase to let users specify where the buttons are on their phone, or we can do an extra CV pass on the battle screen to detect where the buttons are (match the HP and NP text left of the bars to find where a cluster of buttons for a servant). Do you think this approach is feasible?

[MathewSachin]

@thai-ng

a "training" phase to let users specify where the buttons are on their phone

There are many more coordinates other than the skill ones. Specifying them all would be very bothersome.

we can do an extra CV pass on the battle screen to detect where the buttons

This is what I've been trying out for the past few days, but, it's not only the Battle screen that needs to be adapted. Similar logic needs to be developed for all screens.

---

I've changed some things like dialog click locations to be relative to the centre because I believe they're horizontally centred.

It's best if we can figure out their scaling algorithm and just calculate the coordinates but that again needs a good amount of analysis. SeibahMaster on GamePress has collected screenshots of different screens at different resolutions which is really helpful to understand this:

https://community.gamepress.gg/t/automatic-farming-app-fate-grand-automata/72155/2211 https://community.gamepress.gg/t/automatic-farming-app-fate-grand-automata/72155/2220

[neobenedict]

If you don't mind using a modded FGO apk (and accept all the risks that come with that), you can give yourself a role in this server https://discord.gg/8JeHHvh and download a mod which forces the game to 16:9 plus a modified FGA. This will only be available for this lotto, hopefully a fix for FGA is implemented soon.

[yunamon]

Hijacking this thread since the problem is similar.

The latest 1189 Canary kind of botched the scripts on my end. Still works prior to this APK update.

Battle scripts no longer work, Gift box works partially (Can scroll but not selecting any EXP cards) and the roulette box works fine.

[reconman]

@yunamon So how does your problem relate to JP and 16:9 screens?

[yunamon]

@yunamon So how does your problem relate to JP and 16:9 screens?

Well, it's JP. And the script problems since the wide screen update basically causing them to act weird? Gift box script feels suspiciously like the widescreen problem.

[reconman]

@yunamon Nothing has been done to address this problem yet, so of course stuff is still not working if you have a non-16:9 display. If you have a 16:9 display, then it's a different story.

[yunamon]

The problem is the scripts totally doesn't run. I don't see any touch registers. Gift box partially works since I can see the scrolling, but no clicking of EXP cards (Still works prior to 1189).

This only happens on 1189 AFAIK. I used previous versions of Canary with no problem until now. Might want to have a look at that APK.

[yunamon]

The problem is the scripts totally doesn't run. I don't see any touch registers. Gift box partially works since I can see the scrolling, but no clicking of EXP cards (Still works prior to 1189).

This only happens on 1189 AFAIK. I used previous versions of Canary with no problem until now. Might want to have a look at that APK.

Can confirm 1190 fixed the problems I mentioned.

[MJAbeita]

I am somehow able to run battles but not able to utilize any Mystic Codes on my Samsung Note 20 Ultra on Android 10. I had to mess with my phone settings within 'Develer Options' using the 'Simulate display with cutout' setting. For my phone, I used the 'Double Cutout' setting. This reduced and centered my in-game screen to the point where some stuff works from choosing skills, support, servants 1, 2, & 3, NPs, but not the Mystic Code skills as the tap itself for the swapping Mystic Code is unable to properly tap on the skill (It is just short of tapping it but tapping between the skills, i.e. between Skill 2 & 3). This should be the same for other Mystic Codes, I believe. I haven't tested other waves beyond Wave 1, Auto Re-Battle or Apple resets.

Otherwise, I am able to properly initiate a run but supposedly cannot finish a run using Mystic Codes, as far I have seen.

Phone: Samsung Note 20 Ultra Android Version: Android 10 Aspect Ratio: 19.3:6 Resolution: 3088x1440p (not including the double display cutout setting)

[dosudosu]

Just tested JP with canary-1219 on Samsung S10 Lite (1080 x 2400, 20:9) and it seems to be working perfectly so far. Thanks for your hard work @MathewSachin

[Tamonten94]

Everything works perfectly fine. This case can be closed now. Thank you for the new UI Update! Tested JP with Samsung Galaxy S10e

[buitim]

Occasionally running into this issue with Canary 1220. FGA will open the (leftmost) servant's details then fail to recognize cards on the screen, as there are none.

Xiaomi Mi 9T with PixelExperience on A10

[reconman]

@buitim Any chance you can do a video recording with debug mode?

[potchy]

@MathewSachin I am very much interested in the technical details about how this problem was/is being solved. I looked at the code and, correct me if I'm wrong, but I assume these new extension methods xFromCenter, xFromRight, etc. work like anchors. You assign an edge of the screen for every Location/Region so that their positions are adjusted according to the screen size, kinda like this:

I admit I didn't research much about how JP's new scaling works, but I assume DW did something similar to this. There are also some if (isWide) statements I've found in the code, but I'm not really sure how they are different from the new methods. Anyway, when things calm down a bit, I would really appreciate a small write up about your experience with this. Thanks a lot as always. :satisfied:

[buitim]

@buitim Any chance you can do a video recording with debug mode?

Sure thing! I also found out that this might only happen when there is a "dangerous" enemy or a servant (in this case, a servant) and the setting to target them is enabled.

The video is on Streamable here.

[MathewSachin]

@buitim what are those black bars on the left and right of the screen? IIRC someone said it was for gestures. When an already targetted enemy is clicked, the details box pops up. FGA clicks on the top-right corner of the screen to close it in-case it does show up. But, for your device, that location is on that black bar which doesn't get recognized by FGO leading to further strange behaviour. Those black bars would also shift coordinates which could potentially cause other matching problems. See if you can turn them OFF.

@potchy you're correct about xFromRight, xFromCenter, etc. The most important thing is they have 2 different sets of layouts. A static layout and a dynamic one. The static layout is 16:9 like before the UI update and is used up to 18:9, which means without any change, FGA would continue to work up to 18:9 screen sizes. This was found out by SeibahMaster on Gamepress. I've linked their posts above which were really key to the solution. Also, screens less wide than 16:9 still have blue-bars on the top and bottom. This means that the vertical coordinates do not need any changes unless they changed the location in UI. For sizes greater than 18:9, most things are either right/left/centre aligned. Dialog boxes are centre-aligned. Skills and Enemies on Battle screen are left-aligned. Master Skills and Attack button are right-aligned. They aren't scaling the UI components, just aligning them. Now, isWide checks whether we're on JP and the screen size is wider than 18:9. This allows us to share the existing code and still keep it working for other servers. We need this because the coordinates vary depending on whether it is the static layout or the dynamic layout. One peculiar alignment is on Support selection screen, where instead of centring on the screen, they centre it between a part of it.

I'm not completely sure yet if this works for all screen sizes since I haven't tested it out that much. And, I still believe they'll change the UI as they said in their notices, hopefully for better.

[buitim]

what are those black bars on the left and right of the screen? IIRC someone said it was for gestures. When an already targetted enemy is clicked, the details box pops up. FGA clicks on the top-right corner of the screen to close it in-case it does show up. But, for your device, that location is on that black bar which doesn't get recognized by FGO leading to further strange behaviour. Those black bars would also shift coordinates which could potentially cause other matching problems. See if you can turn them OFF.

Yeah, I think those are from the gestures on stock Android ROMS. Didn't have that when I was using MIUI IIRC. When switching back to 3-button layout, FGA works perfectly!

[weitecklee]

I'm not sure if the gift box script in JP is still being worked on or not but it's not working for me (Samsung Galaxy S9, 2960x1440, tested with canary release 1246). Please disregard this if it's still in work. Everything else is working great, thanks for the hard work!

(Ignore the back half of the video, that was me trying to get the script to stop)

https://user-images.githubusercontent.com/64213090/103290554-d2d5c580-49b7-11eb-8575-8d91200c0555.mp4

[MathewSachin]

@arkteck try the latest canary now. I was indeed working on it.

[weitecklee]

Yep, it's working perfectly now! Thank you so much!

[Kubrivsky]

Im sorry, where can i get the new canary?

[the3dsandwich]

@Kubrivsky You can find the link to the FGA canary from the FGA website, which is linked from the description of this repo (README.md).