improve performance via imageSearch

[FanghanHu]

Hi Lailloken, I'm wondering if you could change the scan process to use a screenshot instead of live search on the screen, the performance would be much better, and the user experience would be better as well, you will be able to start a scan, and continue to fight monsters.

https://www.autohotkey.com/docs/commands/ImageSearch.htm

if that is implemented, you could also add a config to enable auto-scan: Whenever an altar is opened, a screenshot is taken, and the scan begins in the background, a progress bar appears on the side (xx/64). once the scan is finished, new scans will not be triggered in a set amount of time(10 seconds maybe?), during this time, the player can open an altar and the first priority receipt is highlighted automatically, (if no priority receipt is available, then a burn receipt is highlighted).

I understand that you probably have a lot of other things to do, and you have said that you will not be adding more features, but I just want to put this idea out there, I appreciate your time and effort in this project.

[Lailloken]

I understand that you probably have a lot of other things to do, and you have said that you will not be adding more features

Sorry, I meant I don't have any features planned as in I don't have any ideas left of what else could be implemented to improve archnemesis, but I'm open for suggestions if someone else has ideas.

I'm wondering if you could change the scan process to use a screenshot instead of live search on the screen, the performance would be much better

The scanning system is indeed using the ImageSearch command, but I don't think it can be used the way you described. As far as I can tell, it can only be used to analyze if an image file is present on the screen, not if an image file is present in another. The scanner also uses a kind of snapshotting to compare the previous inventory state with the current one, which means scanning speed is faster the fewer differences there are between the states. The intended use-case is to scan before entering maps, e.g. while map portals are opening, and to plan ahead. There should always be enough in the inventory to run at least one recipe or two burners. If not, just do the map and either scan after looting two or three mods or after the map.

I had thought about auto-scanning or background scanning in the past but never really got it going since there is no reliable way to tell what makes a scan fail -- the user prematurely closing the inventory before the scan has finished, a new icon not specified before occurring, the mouse hovering another mod making a tooltip cover the inventory, etc.

Additionally, there's the problem of the tool not knowing whether a started recipe has been completed or not. If you click a three-component recipe, it gets highlighted in the inventory and generally stays like that unless you leave the map or the client closes/crashes. If that happens and the scanner has already updated the inventory state and suggestions before finishing the recipe, it will notice one or two components missing and remove the previous suggestion since it doesn't meet the requirements any longer. Then the user has to look up the recipe somewhere else if they can't remember the remaining components, which becomes more likely if it was just a burner recipe the user doesn't really care about. The in-game recipe indicators (not the 3.17.1 filters, the frames) may help out in situations like that, but it's not really an elegant fallback solution.

[FanghanHu]

sorry, I misunderstood the function, upon further research, I found this custom script that does an ImageSearch on a file: https://www.autohotkey.com/boards/viewtopic.php?t=18521

as for ways to tell player that something failed during auto-scan, I think a prompt saying "auto-scan failed, please start a normal scan with the altar open" is good enough.

[FanghanHu]

Additionally, there's the problem of the tool not knowing whether a started recipe has been completed or not.

That's a good point, maybe only auto-scan once per map would solve it?

[Lailloken]

I found this custom script that does an ImageSearch on a file

Interesting, I will look further into this once I think of ways around the aforementioned issues.

[Lailloken]

I played around a bit, and out of curiosity, how long does scanning typically take on your end? I did some testing: re-scanning the same inventory state takes about half a second, re-scanning after shuffling eight slots about two seconds, re-scanning a fully-shuffled inventory about 13 seconds. So on average it takes around 200 to 250 ms per changed inventory slot.

[FanghanHu]

that's about right, but I like to zoom though map, so I usually have mods I wanted to use lined up for a couple of maps, then use them up without rescanning. My scans usually take 5s.

Fanghan Hu

On Mon, Mar 7, 2022, 5:23 AM Lailloken @.***> wrote:

I played around a bit, and out of curiosity, how long does scanning typically take on your end? I did some testing: re-scanning the same inventory state takes about half a second, re-scanning after shuffling eight slots about two seconds, re-scanning a fully-shuffled inventory about 13 seconds. So on average it takes around 200 to 250 ms per changed inventory slot.

— Reply to this email directly, view it on GitHub https://github.com/Lailloken/Lailloken-UI/issues/40#issuecomment-1060565590, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYPCF5LYPWQH6YF6AFS67LU6XRKLANCNFSM5P75J6ZQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[Lailloken]

Alright, now I understand how background scanning would be beneficial. Queueing archnemeses for multiple maps never crossed my mind. This also limits the problem cases that I was concerned about and shifts the cost/benefit ratio quite a bit. Assuming I get things working the way I brainstormed, the background scanner would lag behind no more than one map. The latest update would date back to the last time the search field in the inventory was blank, which is usually the first time it's opened after the start of a map -- that would be the only time to take a usable screenshot.

[Lailloken]

v1.22 now has improved scan speed and basic background scanning. I will explore possibilities of auto-highlighting once I know the new scanner is stable.

[FanghanHu]

nice, I will give it a try after work tonight.

Fanghan Hu

On Tue, Mar 8, 2022, 12:57 PM Lailloken @.***> wrote:

v1.22 now has improved scan speed and basic background scanning. I will explore possibilities of auto-highlighting once I now the new scanner is stable.

— Reply to this email directly, view it on GitHub https://github.com/Lailloken/Lailloken-UI/issues/40#issuecomment-1062100573, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYPCFYERHEFBHC3P4GCBTLU66PIHANCNFSM5P75J6ZQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

[Lailloken]

I found some problems with auto-highlighting. The one I mentioned earlier: the script doesn't know when a recipe is completed and also can't reliably tell from counting how often the inventory has been opened/closed, so it wouldn't know when to highlight the next one. I was going to circumvent this by adding an 'auto' button to the letter bar that would highlight four components (2+2, 3+1, 2+1+1, etc.) based on what's ready, so one could just press that at the first altar and zoom ahead. But here's the catch: There are a few constellations that could ruin recipes that are ready, i.e. combinations of recipe components and burners that assemble into different recipes. For example, recipe AB and burner C being ready: the inventory would highlight the three mods, but the user might do A->C->B, resulting in a potential recipe AC being assembled. I don't really know how the in-game recipe indicators would handle these situations, but counting on a 50:50 working out seems unnecessarily risky to me. Putting in cross-checks to prevent these constellations sounds like too much work considering it's just for convenience and saving a few clicks. Is my assessment of these problems correct or have I overlooked something?

[FanghanHu]

That is correct, having multiple recipes that use the same mod is a problem.

Maybe we can avoid it by showing one recipe at a time, but then we need to scan what's already on the altar and move to next recipe if the current one is complete. Is that possible?

On Wed, Mar 9, 2022, 9:43 AM Lailloken @.***> wrote:

I found some problems with auto-highlighting. The one I mentioned earlier: the script doesn't know when a recipe is completed and also can't reliably tell from counting how often the inventory has been opened/closed, so it wouldn't know when to highlight the next one. I was going to circumvent this by adding an 'auto' button to the letter bar that would highlight four components (2+2, 3+1, 2+1+1, etc.) based on what's ready, so one could just press that at the first altar and zoom ahead. But here's the catch: There are a few constellations that could ruin recipes that are ready, i.e. combinations of recipe components and burners that assemble into different recipes. For example, recipe AB and burner C being ready: the inventory would highlight the three mods, but the user might do A->C->B, resulting in a potential recipe AC being assembled. I don't really know how the in-game recipe indicators would handle these situations, but counting on a 50:50 working out seems unnecessarily risky to me. Putting in cross-checks to prevent these constellations sounds like too much work considering it's just for convenience and saving a few clicks. Is my assessment of these problems correct or have I overlooked something?

— Reply to this email directly, view it on GitHub https://github.com/Lailloken/Lailloken-UI/issues/40#issuecomment-1063056589, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAYPCF24UMYDWIOH37DJCG3U7DBKPANCNFSM5P75J6ZQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

Fanghan Hu

[Lailloken]

we need to scan what's already on the altar and move to next recipe if the current one is complete. Is that possible?

The script could try to track changes in the inventory and deduce from that if a recipe is completed, but that is kind of inconsistent because one may find matching components in between and background scans only work while the search box is blank anyway.

I had another idea: I could add a checkbox next to the scan button which would enable auto-suggestions after each scan has finished. By reading the search box contents before clearing them, the script would be able to tell which and how many mods may still be added to the altar. It would only take three clicks (scan, ctrl-click mod, activate encounter) to run an altar, and the scans wouldn't take long since the inventory states would only differ slightly since refreshes are so frequent this way. I don't know if there is much left that's feasible to make archnemesis be automated or more time-efficient.

[FanghanHu]

If we don't know what's already on the altar for sure, we can't really have auto highlighting.

I had another idea: I could add a checkbox next to the scan button which would enable auto-suggestions after each scan has finished. By reading the search box contents before clearing them, the script would be able to tell which and how many mods may still be added to the altar. It would only take three clicks (scan, ctrl-click mod, activate encounter) to run an altar, and the scans wouldn't take long since the inventory states would only differ slightly since refreshes are so frequent this way. I don't know if there is much left that's feasible to make archnemesis be automated or more time-efficient.

That could be the next best thing to do.

[FanghanHu]

Good job with the update, I'm now in my laziest form, thanks a lot! Do you have a donation link or something, I want to buy you a beer/coffee :)

[Lailloken]

Glad you like it. Thanks for your suggestions, they were fun to work on and improved performance by quite a bit. I really appreciate the gesture, but I don't take donations as this is just a fun project and learning experience. People being interested in and using my script is rewarding in itself.