Neural nets

[Shadowcoldzx]

This is a pretty convoluted and misleading topic, but I couldn't find anything online on this and you seem to have a pretty good grasp on the underlying stuff.

Do you reckon one could build a neural net and create bots to be more or less "human like" in terms of combat?

There are several problems that come to mind:

• how would one get AI to train on WoW
• is there any API beyond lua (probably, considering how current playerbots do)
• how do you get to train the AI
• how would you train the AI in terms of rewards, how do you reward an AI in dungeons, how do you reward it in arenas, how does it get to know the need to do quests / grind / etc.
• while presumably needing thousands and thousands of training hours
• while also needing tanks to rush through dungeons, healers to prioritize cds/consumables, dps to actually aoe/cc as per individual circumstances (rather than a >1 enemies and check again 5 seconds laters) -etc.

My main issue with bots in WoW in general is that they are mostly scripted - they don't really do much else but follow a script, which obviously takes you only so far.. It makes do, but it's heavily predictable and very much prone to flaws.

I get that the point of SPP is to provide a kinda of "offline" experience kinda like an online replica but with AI - and not to replace humans per se, but the term AI is very overused and when we look at the complexities of MMO, it falls down the roof very quickly. It is way too "script like and flaw like".

I guess my over-arching question is, if you were to make a local WoW server with up to 40 or more bots to do bgs/raids/arenas with, how would you approach it? Do you think it impossible, or possible. If possible, would you rely on any of the underlying bot libraries, or would you just rather train AI on pixels from the scratch, if so, what would the goals be. And so forth. What do you currently see as a viable approach.

I fear the answer to all of this is "depends", "not currently known/possible" or "that's not how it works". Though as much as I hate online, I'd always wish to have actually capable bots. The problem is that they need to be "actually capable", i.e. capable of 3k live and such. I myself am a multi glad and regular bots are too slow in dungeons, too dumb and scripted in bgs and arenas. The current bots are simply too scripted, too slow, too predictable, too stupid - I need these bots to have some redeeming qualities, else they are are just trash, predictable trash. Basically, I do not imagine any scripted AI to ever fully replace the online experience (let's be real, most people online are worse than bots..). It is just an aim.

I cannot help but think that there isn't any good way of doing this, not in this game. I just don't see any API-way of getting thousands of agents playing against thousands of agents and competing for dungeon clear times or arenas wins or world pvp or battleground or etcetera.

Then again, you have much better understanding of this, do you see any such way possible? It doesn't have to be realistic, more of an anything goes kinda thinking. In abstract, I think I can only imagine agents training on actual "screens" understanding pixels, but not much beyond that, nevermind any leveling or rewards or dungeons or whatnot. Tough.

[Shadowcoldzx]

That said, I wouldn't mind taking a closer look at the current AI's decisions, but it is very apparent it follows a script, hence it can never truly replicate the "best effort possible" or "who cares" kind of human logic. It does predictable things at predictable times, hence, it is not sufficient for anything more than most basic local sever capabilities. You cannot raid with it, auctions are kinda random-ish (AHBot), you cannot really dungeon with it, you cannot really bg/arena with it, it's just a bunch of scripted behaviours that only meet the most basic "this thing exits" quota. I wish for more, but I also understand that there are a lot of complexities to go along with it and that WoW (especially 1.x - 3.x) wasn't made with AI agents in mind so all that I have left is thinking "external pixel-based shit".

[Shadowcoldzx]

Just to clarify, I'm not as much as about "this exact way", I'm more of about "this would be the way", if that makes any difference. It is a pickle..

[celguar]

people do dungeons and raids. ofc you need to control bots

[celguar]

source code is open btw https://github.com/cmangos/playerbots

[celguar]

for raids the usual approach is make bots follow tactics, that works and there is a server that makes bots do all vanilla raids on their own. No AI is needed for that, no AI is needed to make bots fast, they interrupt you after 0.1s of casting. If you give me a better example of what you want bots to do, we can discuss

[celguar]

is there any API beyond lua (probably, considering how current playerbots do) current bots are written in c++ and are a part of the server.

per individual circumstances (rather than a >1 enemies and check again 5 seconds laters) all that can be coded, there is no need for AI to code it. You just need to define "individual circumstances"

[Shadowcoldzx]

Ah well I've been a little tipsy there haven't I, thanks for the suggestions - though that is the kind of thing I'd be hoping to avoid to get the bots feel more natural - i.e. I find it practically impossible to define "individual circumstances", not without coming up with them for 10 years and intentionally randomizing it to an arbitrary extent (to reduce some of the predictability). Even then it would be predictable and bad.

It's fine in the general sense and fine for most people, it's just not good when you've seen all and done all, then the bots feel more as pets that you can predict exactly what they will do (if we don't count being too slow as unexpected) and they don't excel at anything, so personally I would either just try to solo content with customized solocraft scaling or only bring 1 or 2 more playerbots (typically just healers). When you start needing to control the bots individually then it feels like too much puppeteering rather than playing the game.

I think the conclusion is that it's fine in general for the project type (and it kind of amazes me how much time people such as you spend on it over such a long time), I was thinking of something trainable that does not use scripted conditions, rather something that does things on it's own, in a much more fluent manner - which could be a game on it's own entirely just training it. I guess you could imagine something similar to Alphastar when it was trained to play dota or sc2 - as an AI agent that has learned to play the game, rather than follow a flawed/abusable script. (Limiting it to instanced areas might be more realistic than the world which might be too complex.)

The playerbots could probably be a solid base for it to start with and I suppose given a ton of time one could perhaps figure out a different way of handling the AI other than giving it rigid if/else instructions - I think initially I was more of "hmm where to start lol" but I suppose I have an idea now, digging through the playerbot repo a little bit. (Thanks! I had a wrong conception of how it worked which feels embarrasingly funny in hindsight.)

Imagine the problem with conditions as this:

#Bot.Update() {
  /* The bot must know where it is, what spec it is, what it's supposed to be doing, who he is with, etc., a TON of information omitted from this condition entirely! 
  For demonstration purposes, the bot is a warrior made for 3.3.5 in party of 2 and about to be engaged by other 2 players(bots) in the world
  */

  // what classes are the threatening players? we can probably skip this for ease of use / getting the world more active instead of just running
  // are we outgearing them or are we severely undergeared, in combination with the classes? we can probably skip this for ease of use / getting the world more active
  // do we have sufficient health and mana in comparison with the opponents to engage or try to disengage? You can't script this without the bot having an omega-comprehensive value tables (which are bound to be incorrect) and doing some heavy calculations during this update
  // if we choose to engage, how?
  // how do we change strategies moment to moment based on what is happening - am I able to intercept the mage so my hunter can scatter trap the paladin, are our positions feasible to do this, how do I react if the mage blinks, if he blinks, did deep wounds proc - can we scatter the mage and zerg the pally, does my hunter have scatter and ss cooldowns up? Should I bladestorm for pressure, or should I keep it for poly/hoj protection or to remove slows so I can kite if needed (if I expect to need it in this particular scenario), what if my hunter is a dummy player and decides to zerg the mage, should I follow up and change my strategy, or do I go on the paladin instead? What if the mage does not blink and decides to sit it out.. What if he gets HoPed. What if the mage trinkets scatter, we can no longer engage anything there and we need to get out NOW.
 // suppose we chose to tunnel the mage, now the paladin is casting holy light on max range
 // - is the mage's help low enough for me to interrupt the paladin with a charge? is my hunter in a position to capitalize on this (ideally kill the mage or advance us further?), if the paladin is out of range, is it low enough for me to get in range (taking into account any chill effects) before the cast goes off? should I instead wait for silencing shot? is the mage blocked, should I get away to avoid nova and get in charge range?
 // ... etc ...
}

There are so many, so many, so many variables that cannot be coded, and at some point, performance may become a significant concern, as the bot must understand so many aspects of the game for it to not feel useless. In the given (laughably simplified!) scenario, with current playerbots, one bot gets sheeped the other gets nuked in HoJ nobody drops below 90% hp. With something similar to a neural net, you might lose the fight if the warrior/hunter execute an actually feasible strategy, and understand the importance of cooldowns, positioning or the need to kite (such as waiting out your cooldowns during which they cannot reasonably engage without flat out losing, such as pre-popped icy veins / frost finger stacks from a mob, whatever). That is my point against scripted circumstances - none of this can be realistically coded.

I'll close this because this isn't really related to SPP at this point, but wanted to elaborate a bit on what I originally meant.

[celguar]

I don't believe a trainable bot can even come to conclusion that it needs to run away from Baron Geddon bomb so that it doesn't damage the raid. Not speaking of even AQ or naxx. Feel free to prove me wrong though.

[celguar]

And yes we can script bots thinking if its outnumbered or not. There is already a code for that, you just need to invest much more time in it to cover mot cases. I don't believe an AI would do better here, you would have to teach it to look at NPCS spells, damage, attack speed, hp, amount of NPCs and by that you already have a formula to use without AI.

am I able to intercept the mage so my hunter can scatter trap the paladin, are our positions feasible to do this, how do I react if the mage blinks, if he blinks, did deep wounds proc - can we scatter the mage and zerg the pally, does my hunter have scatter and ss cooldowns up? Should I bladestorm for pressure, or should I keep it for poly/hoj protection or to remove slows so I can kite if needed (if I expect to need it in this particular scenario), what if my hunter is a dummy player and decides to zerg the mage, should I follow up and change my strategy, or do I go on the paladin instead? What if the mage does not blink and decides to sit it out.. What if he gets HoPed. What if the mage trinkets scatter, we can no longer engage anything there and we need to get out NOW. I am sure this can be simplified to 5-10 conditions and triggers. And I Don't belive a real human would think all that in hot fight without preparation

[celguar]

And of cource all that is 1000x simpler that teaching AI all that (but first make it get all that info real time from server = write an API that AI would use to get the info = ai delay huge = bots slow. Meanwhile all that you wrote can be written in several bot triggers/actions/strategies in c++ code very fast. But is it worth it, considering they need more common use improvements instead? I guess no. If they were perfect in general situation, then might be worth coding it. If you beleive and know how to make AI to write all that instead, feel free to explain