[Feature Request] Machine learning AI bots

[forest0xia]

Ultimately, the bots play style is static/fixed with the current AI approach provided by Valve at the moment. We need machine learning AI bots!

We are also looking for a better support for the scripts in all aspects from Valve. And want to know if Valve is ever going to improve the bot performance again.

1. The OpenAI-Five ( https://openai.com/index/openai-five/ ) was completed so many years ago, is there a date we will be able to play with similar ML AIs again?
2. will there be a time Valve improves the default script empowered by the ML AIs and other developers can do further improvement later?
3. Is there any other good sources we can use for training AI bots already?
4. How much money/time for the hardwares would it cost to train a model against the current Dota2 version?
5. How much money/time for the hardwares would it cost to keep training or improving the model for new Dota2 versions when released?
6. What kind of hardware is needed minimum to run the model for a 5v5 game play. Is it a normal PC, or can a normal person affort such a pc.
7. How much effort from developers, how many developers would we need to build good AI models. How do we pay the developers?
8. How many models would we need. E.g. do we need a model for each different behavior modes?
9. TBA

[forest0xia]

Think most questions we may have should have been answered by the authors of https://openai.com/index/openai-five/, but unfortunately the public resource of reproducing the work is very limited. As we are having more advanced AI technologies nowadays, not sure how much we can simply or reduce the cost of making such an AI.

[christiandwi]

@forest0xia I want to discuss here too, but i could be wrong here as i'm not an expert in creating bot or AI

2. Up until now, i don't see any notable improvement to the bot scripting from valve, they broke the bot from 7.33 April 20, 2023 on the neutral token, they didn't even care about how bot behave on picking and choosing neutral token and i assume that there is no such thing on valve bot API. 1 year with no improvement means all hope is lost here. Co-op bot match script is only for selected scripts and not counted on match count, i can rant all of my lists but that's not the point here. The point is valve doesn't care at all about bot on this game.

this specification is not affordable at all from my perspective, and it's not worth it if we are going to open-source this one.

and for item & skills build, this open-ai five only uses hardcoded one, they didn't get from guide / dotabuff. if we want to do this, even if we use OpenDota API to get the item build popularity, the build will still be considerably static because of this example

Meepo build PT -> dragon lance -> blink on 80% of let's say 1,000,000 matches if we hope that meepo build PT -> blink -> dragon lance, we need to wait all top players use that build for more than 800,000 matches, it won't dynamically change on 1 day

so like you said on earlier issue, it's not worth the hassle to concentrate on the dynamic item & skill build

is there any active forum / chat platform beside this github? i would love to contribute here but i don't really understand .lua and the debugging process here, if you can guide me on how to start this, i have some spare times to contribute on this bot.

[forest0xia]

With the old tech stacks/tools (back in 2018) yeah it's not affordable to any individual, for training or running the model. But I think the work can be simplified in a way that we can split the models, so to train a general model for specific behaviors like laning, attacking, timing for roaming, etc. Those behaviors can have a lot of micro controls and are very hard to make good in normal coding practice, and e.g. attacking should be a good option to train a model for melee vs ranged heroes, right now bots are dumbly standing/walking and attacking.

Another point is we do not need to exhaustively train the bots to beat TI pro players. As long as we can make general models perform better than the 2k or 3k players most the time, its a huge win. And it wont require that much of resource usage comparing to what OpenAi Five needed. IMHO.

To the point of dynamic builds, yeah I dont have a plan to do that. And thanks for sharing that even openAI Five team didn't want to do that either LOL, interesting and reasonable to know.

Tho I don't think you understood correctly the approach of what dynamic means in my mind. The dynamic builds should have at least 2 tenets:

1. Can follow the hot builds in the live Dota2 version, contributed by guide makers. Note there are a lot of guides for builds. So there are strategies to follow how one is the best option on this game, in which order to get the most out of the items/abilities at each level. And some items are similar and interchangeable like for escape we can purchase force staff or Glimmer Cape, etc. Can just have some dummy strategies like randomly pick of those similar purpose items. so it won't be static in a game. Even if one item shows 100% winrate, we can have the strategy implemented in a way to add weights to the winrate vs the gold required, number of times played, etc.
2. Can make counter items depends on what enemy heroes have. E.g. MKB vs BF. There are a lot of those, and item properties keep changing, this better be dynamic to pick the best counter item and who in the team is the best to purchase the item - pos1-2 purchase counter items with more dps, and pos4-5 purchase items that's cheaper/realistic for them to get.

I dont have any forum. Github is good to me. Just post questions.

You no not need to know LUA now, just use typescript: https://github.com/forest0xia/dota2bot-OpenHyperAI/tree/main/typescript. It's easier and widely used for different purposes so it's good if you already using TS, and good to learn because there are a lot of good videos for learning TS.

[christiandwi]

Ahh understood, unfortunately i'm not an AI developer so i don't really have the answer for that.

And yes, now i understand too about the dynamic build concept here, that would be great to implement, maybe we can start on the simplest form of this like if enemy is PA or have some evasion, then bot should consider to buy MKB

I was trying to ask about a basic question actually, like how to debug my code here? I tried to use dota_bot_reload_script after each changes, but i don't think it works, or maybe my changes was not good enough. Should i open an issue for this basic question? Noted for this, i will try to implement the script using TS and maybe learn some TS to get a job hahaha thanks!

[forest0xia]

dota_bot_reload_script may not work properly for native bot script on client side, and it shouldn't work since we keep quite some status variables to track bots' setup and some status only take effect in early game stages which can lead to even more problems if we use dota_bot_reload_script to reload and miss those status variables.

If you changed bot's behavior, just reload the entire game by creating a new lobby again.

Haha yeah, one good thing about utilizing ts! Learning to do stuff with joy like playing games is the best way to do things. Kudos to @ThirVondukr for convincing and adding the ts basic layer support.

[andre1891]

There

dota_bot_reload_script may not work properly for native bot script on client side, and it shouldn't work since we keep quite some status variables to track bots' setup and some status only take effect in early game stages which can lead to even more problems if we use dota_bot_reload_script to reload and miss those status variables.

If you changed bot's behavior, just reload the entire game by creating a new lobby again.

Haha yeah, one good thing about utilizing ts! Learning to do stuff with joy like playing games is the best way to do things. Kudos to @ThirVondukr for convincing and adding the ts basic layer support.

I know there is an console command which refered to "AI learning Bots" dota_bot_use_machine_learned_weights

[forest0xia]

@andre1891 I know this command dota_bot_use_machine_learned_weights was available like at least 5 years ago, but there is no documentation of what exactly it is or how it can work with bot scripts. Do you know any more details?