7.0.0 update has new UI style

[dbzdivik]

you need to spend quite a time for sure for this new UI Update along with Dispatch Mission

[tmarenko]

Dispatch Mission will be added later. Fixing current UI elements is a goal for now. It's just a tedious work because some elements just have changed position for no reason at all (thx NetMarble).

[tmarenko]

First iteration done in https://github.com/tmarenko/mff_auto/commit/ad29c5a71f33db799d2e4e5bb5b7137ec73d1195 Still have 47 UI elements that has rare occurrence and need to test.

[geniusdragon]

Thank you for making changes so quickly.

[geniusdragon]

So far the main issue I've seen is that the actual scanning of skill buttons doesn't seem to be happening. For instance, it will go into legendary battles as it should, but then your character will sit there and do nothing. The logs won't show it picking up the skills, and therefore never moves to pressing them. If you queue all 5 stages for example, if you manually press the skills to win the battle, it WILL however move onto the next stage correctly though.

The second thing I've found is that even with full energy and 100 boost, when you attempt a giant boss raid, it will get to the main screen where it shows the rewards and allows you to enter, but will then throw up "INFO - No more stages for Giant Boss Raid."

[tmarenko]

So far the main issue I've seen is that the actual scanning of skill buttons doesn't seem to be happening. For instance, it will go into legendary battles as it should, but then your character will sit there and do nothing. The logs won't show it picking up the skills, and therefore never moves to pressing them. If you queue all 5 stages for example, if you manually press the skills to win the battle, it WILL however move onto the next stage correctly though.

The second thing I've found is that even with full energy and 100 boost, when you attempt a giant boss raid, it will get to the main screen where it shows the rewards and allows you to enter, but will then throw up "INFO - No more stages for Giant Boss Raid."

What's your emulator screen resolution and in-game graphics setting? For me (1920x1080, HIGH graphics) GBR and battles are working fine. My UI tests also shows that for 720p and 1440p all should work fine.

[geniusdragon]

Emulator is 1920x1080 and HIGH as well. I haven't changed anything since it was working prior to the new patches (for the game and the bot obviously). I can attempt a reinstall, as this is with an update via the updater.

[tmarenko]

Try to find settings\ui\battle.json file and open it with any text editor. Find records for SKILL_1_LABEL, SKILL_2_LABEL ... SKILL_6_LABEL and try to change image_threshold parameter for them. For example if image_threshold value is 150 then try to set 140 or 160, 130 and 170, etc.. Then restart bot and try to play a battle. For GBR that would be GBR_MENU_LABEL and GBR_QUICK_START in settings\ui\giant_boss_raid.json file.

If some image_threshold values worked for you let me know.

[geniusdragon]

Odd. I'll try some now, but SKILL_6 is the only one showing anything close to that, 170. Whereas the rest are sitting at 0.7.

[tmarenko]

SKILL_6_LABEL not SKILL_6 Here: https://github.com/tmarenko/mff_auto/blob/master/settings/ui/battle.json#L565-L643 https://github.com/tmarenko/mff_auto/blob/master/settings/ui/battle.json#L328-L342

[geniusdragon]

Sorry, should have said SKILL_6_LABEL. That was what I was referring to.

[geniusdragon]

I haven't found an image_threshold that works as of yet - I'll try some more tomorrow.

[tmarenko]

Try to create tesseract folder inside logs folder. Bot will write images of UI elements into this folder. That will help to understand what bot sees from the screen:

Another way around is to run this script:

import lib.logger as logging
from lib.video_capture import NoxCapture
from lib.players.nox_player import NoxWindow
from lib.game.missions.legendary_battle import LegendaryBattle
from lib.game.missions.giant_boss_raid import GiantBossRaid
from lib.game.game import Game

logger = logging.get_logger(__name__)
logging.create_file_handler()

if __name__ == '__main__':

    nox = NoxWindow("NoxPlayer")
    with NoxCapture(nox) as nox_capture:
        try:
            game = Game(nox)
            LegendaryBattle(game).do_missions()
            GiantBossRaid(game).do_missions(times=1, max_rewards=True)
        except KeyboardInterrupt as err:
            exit()

That script will run Legendary Battle and then GBR and the script will capture video into .mp4 file in logs folder (similar to https://youtu.be/QcgZcAwBL-I )

[dbzdivik]

World Boss Invasion Issue => Stuck at battle over, f.y.i chest receive notification is closed by bot

2021-04-23 21:26:20,182 - battle_bot - INFO - Battle is over 2021-04-23 21:26:22,782 - world_boss_invasion - DEBUG - Any chest after boss fight wasn't acquired.

Alliance Battle => can't start it

2021-04-23 21:24:37,563 - game - DEBUG - Stage: TIMELINE BATTLE; stages: 10/10 2021-04-23 21:24:48,113 - alliance_battles - WARNING - Alliance Battles: can't get in battles lobby. 2021-04-23 21:24:50,225 - missions - DEBUG - After mission notifications was closed: False 202

Also LB is working fine for me

2021-04-23 21:32:00,824 - battle_bot - DEBUG - Casting 3 skill. 2021-04-23 21:32:01,579 - battle_bot - DEBUG - Successfully casted 3 skill. 2021-04-23 21:32:02,927 - battle_bot - DEBUG - Casting 2 skill. 2021-04-23 21:32:04,570 - battle_bot - DEBUG - Successfully casted 2 skill. 2021-04-23 21:32:07,839 - battle_bot - INFO - Battle is over 2021-04-23 21:32:07,840 - missions - DEBUG - Clicking REPEAT button with UI Element: LB_REPEAT_BUTTON. 2021-04-23 21:32:09,660 - missions - DEBUG - After mission notifications was closed: False

[tmarenko]

Just to be sure, can you check if all .json files in settings/ui have same content as in https://github.com/tmarenko/mff_auto/tree/master/settings/ui It's very odd because WBI and AB are working fine for me and have stable UI recognition. Maybe updater failed to change these files, I'm not sure.

If files are the same then: For Alliance Battle try to change image_threshold for AB_NORMAL_READY_BUTTON to 150 or 120. For World Boss Invastion try to change image_threshold for INVASION_HOME_BUTTON to 0.5 or even 0.4 .

[geniusdragon]

Ok, it doesn't even seem to scan the skill labels yet. They're not appearing at all in the tesseract log. I'll have a search through in terms of the GBR to see where it's failing.

[tmarenko]

Ok so here is another script for you:

import lib.logger as logging
from lib.players.nox_player import NoxWindow
from lib.game.game import Game

logger = logging.get_logger(__name__)
logging.create_file_handler()

if __name__ == '__main__':

    nox = NoxWindow("NoxPlayer")
    game = Game(nox)
    from PIL import Image, ImageDraw
    image = nox.get_screen_image()
    image = Image.fromarray(image)
    draw = ImageDraw.Draw(image)
    for skill_index in range(1, 7):
        rect = game.ui[f'SKILL_{skill_index}_LABEL'].rect
        box = (rect.global_rect[0] * nox.width, rect.global_rect[1] * nox.height,
               rect.global_rect[2] * nox.width, rect.global_rect[3] * nox.height)
        draw.rectangle(xy=box, outline="#00ff00", width=4)
        print(f"Value from SKILL {skill_index}: {nox.get_screen_text(game.ui[f'SKILL_{skill_index}_LABEL'])}")
    rect = game.ui['MELEE_BUTTON'].button
    box = (rect.global_rect[0] * nox.width, rect.global_rect[1] * nox.height,
           rect.global_rect[2] * nox.width, rect.global_rect[3] * nox.height)
    draw.rectangle(xy=box, outline="#00ff00", width=4)
    for tr in range(1, 11):
        threshold = tr / 10.0
        game.ui['MELEE_BUTTON'].threshold = threshold
        print(f"Melee button is on screen: {nox.is_image_on_screen(game.ui['MELEE_BUTTON'])} "
              f"with threshold={threshold}")
    image.show()

Open any battle with your character (skill preview f.e.) and run it. You should see screenshot similar to this:

and log file should containt some info like this: Value from SKILL 1: 1 Value from SKILL 2: 2 Value from SKILL 3: 3 Value from SKILL 4: 4 Value from SKILL 5: 5 Value from SKILL 6: 6 Melee button is on screen: True with threshold=0.1 Melee button is on screen: True with threshold=0.2 Melee button is on screen: True with threshold=0.3 Melee button is on screen: True with threshold=0.4 Melee button is on screen: True with threshold=0.5 Melee button is on screen: True with threshold=0.6 Melee button is on screen: True with threshold=0.7 Melee button is on screen: True with threshold=0.8 Melee button is on screen: True with threshold=0.9 Melee button is on screen: False with threshold=1.0

[geniusdragon]

Didn't log, but I added a pause to check it.

Value from SKILL 1: 1 Value from SKILL 2: 2 Value from SKILL 3: 3 Value from SKILL 4: 4 Value from SKILL 5: 5 Value from SKILL 6: 6 Melee button is on screen: True with threshold=0.1 Melee button is on screen: True with threshold=0.2 Melee button is on screen: True with threshold=0.3 Melee button is on screen: True with threshold=0.4 Melee button is on screen: True with threshold=0.5 Melee button is on screen: True with threshold=0.6 Melee button is on screen: True with threshold=0.7 Melee button is on screen: False with threshold=0.8 Melee button is on screen: False with threshold=0.9 Melee button is on screen: False with threshold=1.0

[tmarenko]

Melee button is on screen: True with threshold=0.7 Melee button is on screen: False with threshold=0.8 Melee button is on screen: False with threshold=0.9

Doesn't look like 1080p to me. But it's not a big deal, I'll change threshold for 0.7

[tmarenko]

Second iteration done in https://github.com/tmarenko/mff_auto/commit/1a1f174f8e80a7f7b6466e0731819cc94f20e41d Still had 28 UI elements to test (some of them should appear weekly or may be even were removed). Threshold values should be more flexible now for low resolution.

[geniusdragon]

I think I've worked out my issue at least - and I feel a bit stupid. I've always had mine nox set on mobile phone - and it was fine previously. I've updated it to tablet, which obviously changes the two h and w figures around for proper FHD.

Would there have been changes to anything that stopped it working on mobile phone mode vs tablet mode in between the updates?

[tmarenko]

I think I've worked out my issue at least - and I feel a bit stupid. I've always had mine nox set on mobile phone - and it was fine previously. I've updated it to tablet, which obviously changes the two h and w figures around for proper FHD.

Would there have been changes to anything that stopped it working on mobile phone mode vs tablet mode in between the updates?

Sorry, I'm not sure if I understood correctly. Are you talking about this settings? Are you saying that bot was working fine in mobile phone mode? And now it only works in tablet mode?

[geniusdragon]

Yes, that exactly. I am getting some errors, but I was able to successfully run alliance battles, and danger room. GBR seems to work as well.

[tmarenko]

I've tried this mobile phone mode and saw no difference (yeah H and W are swapped but when you run MFF app emulator changes orientation to 1920x1080 so it works basically the same). I believe tablet mode is a default mode when you install NoxPlayer, any reason why you changed to mobile mode?

[geniusdragon]

I've had it as mobile phone mode mainly because of a previous bot, but also because it took up less screen space really.

[tmarenko]

Well, I'm not really sure but I didn't notice any difference between mobile and tablet modes.

it took up less screen space really.

Usually I'm making NoxPlayer fullscreen and doing Alt+Tab into another window/application. Bot doesn't need to NoxPlayer be actually on screen in order to get image/send clicks. As long as NoxPlayer is not fully minimazed bot can "see" trough other windows. Making NoxPlayer's actual window lesser size results into similar problems when you set up less then 720p resolution.

[geniusdragon]

I have started doing that, and it does seem to help.

One message I have seen the bot recently get stuck on is when you have the maximum number of items (say crystals), but you're still farming for something else in that mission. The bot currently doesn't pick up the error message for that, so it gets stuck. I'll get a picture for you tomorrow.

[tmarenko]

I'll get a picture for you tomorrow.

Fullscreen screnshot of game with this notification would help, thank you. ITEM_MAX_LIMIT_NOTIFICATION is still in "untested" category because I've never encountered it after 7.0.0 update yet.

[geniusdragon]

[tmarenko]

Fixed in https://github.com/tmarenko/mff_auto/commit/c2c7aa2e39e7b6fa89567a49d336afe9b652ee28

[tmarenko]

UI elements that still haven't been tested: KICKED_FROM_THE_SYSTEM - rare notification in World Boss Invasion, couldn't get it. INVASION_FAILED - end of battle situation when you fail in World Boss Invasion. Almost impossible to get because gamemode is so easy nowadays. RANK_UP_NOTIFICATION_1 - ranking character notification at the end of the mission. Looks like game isn't showing it anymore, at least not in usual mission (got similar in Story mode). INVENTORY_FULL - should be same. GBR_FAILURE_DAMAGE_LIST - same situation as with INVASION_FAILED MAIN_MENU_AD_3 - probably was removed. MAINTENANCE_NOTICE - need to wait for the next game's update. BIOMETRICS_NOTIFICATION - no plans for paying again for this. X_GENE_NOTIFICATION - same. TL_LEAGUE_NOTIFICATION - probably was removed. WB_CLOSE_OFFER_AD - probably was removed. WB_CLOSE_OFFER_AD_OK - probably was removed.

Overall, migration to 7.0.0 UI is done.