manny405 / sapai

Super auto pets engine built with reinforment learning training in mind
MIT License
67 stars 21 forks source link

Extensive Battle Testing #5

Closed manny405 closed 2 years ago

manny405 commented 2 years ago

I've evaluated some exotic teams for validity already, as seen in the file tests/battle_tests.py. More tests should be performed to ensure that battles are being faithfully reproduced. Some current questions I have now follow.

  1. With tiger behind a fly, if the tiger is killed by a snake does it spawn one fly or two?
  2. Do a fly's maximum number of actions only trigger once when a tiger is behind the fly, thus giving the fly a total of 6 possible fly spawns? EDIT 20210121:
  3. Where do bees spawn when honey is applied to cricket, sheep, or fly? Which summon gets priority in the case that the team is full?

Any other questions about the correct way to handle complex pet interactions can be put in this issue thread for discussion. Ideally, all questions should be answered by linking to a time-stamp in a VOD demonstrating each scenario.

clsandoval commented 2 years ago

Do badgers damage opponents summoned units?

As far as I recall badgers damage opponent summoned units on death, might be wrong about this one though will check gameplay to make sure. I am sure that badgers do not damage its own summon (bee). I tested the interaction with the engine and currently the badger on death effect does not damage opponent summoned units.

edit: Yep, badgers should damage opponent summons, https://youtu.be/edjkQ-9dkmQ?t=266

To replicate:

Team1 = badger, fish Team2 = cricket,

Current behaviour assuming badger is on 1 hp:

Attack: badger and cricket knock each other out After Attack: badger damages teammate fish but not enemy zombie cricket

Graph: image

manny405 commented 2 years ago

Good catch! And great to see the VOD.

Turns out, my most recent changes cause Badger to not work at all in fights... Badger now at least works in fights on the Main branch.

The interaction that you brought up requires a little more digging. My understanding is that the way that Pet's abilities are activated is via a priority queue based on the attack stat. Meaning, Pets with a higher attack stat activate their abilities first. The interaction shown in the VOD breaks this principle, so there must be something else going on. So some followup questions are:

  1. Are summons treated first in every single case of pet fainting abilities?
  2. Simple example, if an ant and a cricket attack, does the zombie-cricket always spawn before the ant gives its buff even if the ant has higher attack?
  3. Is there any situation that two Pets faint at the same time and any ability takes place before a summon ability?
  4. In another VOD, I saw two identical honey-badgers faint at the same time. There was a seemingly random roll, and the friendly bee was sent out first, without activating the friendly badger damage. The opponents badger damage was dealt to the friendly bee, then the opponent's bee was sent out winning the opponent the battle. If summon abilities on faint were always given priority, then this situation would not follow that principle. Or is it different because bee is a status...?

If summon abilities are always given extra priority when two animals faint at the same time, this can be patched pretty easily. However, because of #4, it seems its not that simple. Let's verify some of these questions before changes are made to the code-base so that things can be done correctly the first time.

The current behavior after the most recent commit for the situation you have given above is the same. However, if the cricket had a higher attack than the badger, then the zombie-cricket would spawn first, then the badger damage would be dealt, and the battle would be a draw.

clsandoval commented 2 years ago

Its possible that the behaviour of a bee summon is different to that of a cricket summon (spawning first regardless of summoning unit's attack stat) I will look for examples in a VOD for numbers 2 & 3.

manny405 commented 2 years ago

Is it possible that the priority is reversed for fainting abilities? That the order of activation goes in the order of animals with lower attack stat?

Also, I just had a 10-win game where the Rhino's knockout ability activated before a 1-up Scorpion. This video link works for me in a Firefox browser: https://user-images.githubusercontent.com/34068188/150562616-012df9b8-32c2-4486-bf9b-2f48ef7536c4.mov

This is very interesting because the Rhino knockout would activate after a rooster death to kill the spawned roosters in succession, right? What about if there was cricket with a bee against a Rhino?

clsandoval commented 2 years ago

Maybe we can compile a list of interactions/priorities to test and get into a versus lobby to test the interactions? I feel like looking over VODs for these specific instances will be slower.

manny405 commented 2 years ago

There's this repository: https://github.com/bspammer/super-auto-pets

From their README, it seems they use the native Super Auto Pets in the browser to run replays. I guess they are injecting code or info somehow and getting SAP servers to run it... This would be the best way. I haven't tried to get it working yet.

BastiaanBekooij commented 2 years ago

I wanna caution you to not spend too much time on this since an upcoming update will change the order system. Quoting from the SAP discord test channel: "- Changed fundamentally the ability ordering. This will fix Honey Badger vs. Honey Badger not resulting in a draw. It also changes a lot of other things and weirdly delayed interactions. It had already been changed in Round 1, but I wasn't sure how to document it. Anyway. It is really fundamental and touched basically everything."

manny405 commented 2 years ago

@BastiaanBekooij Thanks for the heads up! Seeing as it's recognized that the current version is not extremely consistent, let's stick with the method implemented now. If any agents are trained using current code, the state-values can be re-calculated later.

I'll keep the thread open to identify other bugs/errors. However, order-of-operations probably won't be changed until after the patch.