Improve AI

[dnk777]

I have a plan to significantly improve AI. About 20% or planned changes are made, and progress will depend heavily of my load at work, but current changes are already worth to see. My code is in very early stage, and contains well-known bugs (a memory corruption source of what I did not spot yet), but I cant wait to share progress. The code is in development in my fork.

Done:

• Bots never did bunnyhopping at least in 1.5+, fixed.
• Old bot area checks were stupid and caused bots getting blocked in walls (AI_CheckMove just checked a trace from eyes to forward ground, and detected walls as a ground).
• Old bots have dumb target and weapon selection, they used stick to a last attacker. My code uses quite sophisiticated EnemyPool, that tracks seen enemies, last targets and attackers, suggests targets and weapons using complex heuristics based on distance, health status, powerups, direction, gravity and environment, and suggests advance/retreat/inhibit fight tactics. Bots now do weapon combos to finish enemies. Bots now try to spam last seen enemy positions if there is no enemy in view.
• Old bots never evade projectiles due to a bug. Bots now may evade rockets, gunblade blasts, grenades, plasma streams and laser beams. Plasma projectiles are clusterized in rays based on direction and proximity. Bots evade and dash heavily in fight.

Short-term goals:

• Fix my bugs :)
• Introduce more human-like hitscan aiming by calculating speed of change of dot products between look dir and dir to enemy.
• Indroduce "true" strafe-jumping as described in "Strafing theory" by injx, its a 5-minute task itself but depends of fixing bot mouse movement (bots roll mouse like a drunk each second atm).

[adem4ik]

• Bots never did bunnyhopping at least in 1.5+, fixed.

That isn't really a bug, but the feature. Vic decided to disable bot's bunnyhopping for the gametypes with items in order to prevent that issue - https://github.com/Warsow/qfusion/issues/134 - and some more

[dnk777]

Idk is it really caused by bunny, with a planned improved movement it is likely to be mitigated anyway. I suggest you try my fork anyway, even if it is in very early stage, I'm going to make bots a real beasts.

[viciious]

Not to sound discouraging but the Vec3/Vec3Like/Vec3Ref stuff is atrocious and has zero chances to ever get merged.

[dnk777]

Well, please explain how may I fix that

[viciious]

• We already have C functions for vectors, use them instead.
• I'm not getting the Ref an Like parts either. Vector is a simple scalar class, there's absolutely no need to have two additional layers of classes.
• Vector classes if implemented, should be templated anyway.

[dnk777]

Vec3 owns its own plain old array. Vec3Ref wraps a pointer to a plain old array. Vec3Like is a supertype for Vec3's and Vec3Ref's that contains common vector ops. Vec3Like have a pointer to array, but does not know whether it owns the array or not. If vector op needs its own buffer, it always return Vec3. Vec3Ref have been introduced to make a distinction from Vec3Like that may or not own its array. Vec3Ref is quite similar, to, for example, QLatin1String from Qt and designed to work C++ way with mutable references to old code vectors without touching old code. I thought about adding glm library in a project, but decided that introduction of these wrappers is less invasive and used 'em only in /ai internals

[dnk777]

Its my development sandbox, its not going to be merged right now, I did not make a pull request. We'll discuss my results later when I do it completely

[viciious]

Alright. It'd be nice if you could record some botmatches to demonstrate the AI improvements.

[dnk777]

Its very incomplete and bugged atm, but playing it if it does not crash is funny. Planned completion time is the June 1th. I have added quite lots of code having some spare time these days, will be under heavy work-load next months.

[dnk777]

Upd: pushed new code, now bots try to aim like humans.

Old code just added random noize in target position in attacking frames, some sinusoidal noize when firing LG. Bot with a lasergun was a complete joke.

Let me explain changes.

Human hits 100% when target on screen does not move. Real accuracy on moving target is limited by brain and muscular latency. Bots have a source of latency too, the limited yaw/pitch speed used in ChangeAngle. Now bot tries to always keep its lookdir on enemy. We still add some random noize, but not only in attacking frames.

It needs tweaking as everything else, but now bot aim both much more deadly and human-like. https://github.com/dnk777/qfusion/commit/7b760cff1d11e0f5eda13e66d648aba9c8b3ccdb

[dnk777]

There are not critical bugs (can't trigger AddressSanitizer anymore) and I strongly suggest everybody try to play my fork.

Aside from bug fixes a new feature has been added. Now bots try to detect escaping enemies, fast switch weapon if it is possible and hit escaping enemy hard in a single shot. https://github.com/dnk777/qfusion/commit/f537366186e1d8358873bc26b8accf26167053cd

There are not planned major changes in target/weapon selection/combat code anymore aside from minor bugfixes and values/condition tweaking.

Next goal is to improve bot movement, but it is deferred at least at a single month.

March TODO's:

• Track changes in old edict_t fields that may be set by scripts, I am not going to change the script interface.
• Add some kind of pendingLookDir, pendingLookAtPoint, very useful for turning in combat, rj, wall-rj (in future).
• Extrapolate last seen enemy positions and set extrapolated one as pendingLookAtPoint, bots often loose enemy out of view atm

[solidfake]

Hmm, despite the obvious issue of bots sometimes getting "stuck" into walls and loosing their will to hunt enemies, my main issues with bots are: -the constant flickshooting, bots don't aim where they see that the enemy player is, but where they know where he is (minus accuracy depending on skill), it looks extremely awkward when spectating bots as they constantly lock onto every player in sight even if he is 180° behind them. But that maybe needs a bigger rework, as the only skilllevel bots really have now is their accuracy. -They still have extreme issues with everything that involves a walljump to get at, which makes most maps quite bland when you know they will never contest that RA or reach that platform

But I have no say in what gets approved anyway, just a heads up if you still not feel demotivated enough after fixing your mentioned stuff.

[viciious]

I'll check out this branch whenever my time permits.

[viciious]

I can't say I'm totally happy with STL creeping in as well.

[dnk777]

the constant flickshooting, bots don't aim where they see that the enemy player is, but where they know where he is (minus accuracy depending on skill), it looks extremely awkward when spectating bots as they constantly lock onto every player in sight even if he is 180° behind them.

Its just kept old bots behaviour, they do not check even a dot product between look dir and enemy dir, they just use inPVS. https://github.com/Warsow/qfusion/blob/master/source/game/ai/ai_class_dmbot.cpp#L961 https://github.com/Warsow/qfusion/blob/master/source/game/g_utils.cpp#L1728 It will be reworked to lock on only really visible ( = in field of view) enemies, back enemies will be detected when a bot may hear their shots/jumps or see projectiles coming from behind. In this case, I will set a guessed enemy position as a pendingLookAtPoint (to be introduced).

They still have extreme issues with everything that involves a walljump to get at, which makes most maps quite bland when you know they will never contest that RA or reach that platform

I did not touch movement significantly yet, and, yes, its just wrong it its basics atm. While I have allowed bots to bunnyhop, it its not very useful atm, since bots hit walls all the time due to following reasons:

• Bots check obstacles only on contact.
• Navigation path provides only a coarse direction of movement.

Environment will be tested and for a current path segment (-1 nav node back, +1 nav node forward) a detailed, obstacles-aware movement path will be built.

I can't say I'm totally happy with STL creeping in as well.

STL is to be removed from the ai subdirectory aside from things that do not allocate anything (std::max, std::find, etc). Atm the only container used in code is std::deque, I wanted something reliable to start with, it is to be replaced by a fixed-capacity circular buffer.

[dnk777]

UPD: Buggy attempts to introduce advanced bot movement were reverted. To give bots a chance anyway, I have enabled forward bunny for bot clients. Now bots are OK for FFA (not DM, item pickup code remains untouched and it is weak). There are known bug that often bots start to ignore weapon/target choice timeout, but it does not ruin gameplay.

[adem4ik]

Will be there any performance regressions after applying your improvements, @dnk777 ?

[dnk777]

Will be there any performance regressions after applying your improvements

For servers, definitely yes, but there are huge performance reserves anyway and the code is not optimized yet. Bot code runs on server and does not affect clients (if it does not hang a server :)

[dnk777]

There are following issues that prevent implementing really good AI (mainly movement):

• Bot doesn't have a knowledge about a map environment, it is like a blind man that tries to touch environment with a walking stick. This affects not only movement but target/weapon/tactics selection too.
• Current paths that consist of nodes are too restrictive, we have to follow the path nodes and try to complete them. Thats why I have to remove strafe-jumping code I have added once. You may see that in fight bots move much more vivid and properly, since they are not restricted by nodes and trace environment heavily. I want to know that a bot is in a some box volume with a floor and walls, and be able to use any arbitrary movement path as long it is inside the volume and goes from its entrance to an exit.

Thus I wont touch movement code anymore, aside from bugfixes in current code while there are no rich navigation support in a map data. I think about porting AAS code from Q3A, it would take not less than 2 month given my limited amount of time I may spend on modification and poor engine knowledge. AAS runtime is to be ported first, Q3 maps with precompiled AAS may be used for testing.

[viciious]

The code for q3 bots navigation is monstrous. Porting it to qfusion would be a no small feat.

[dnk777]

AAS code builds well together with the qfusion source, the only thing needed is to fix some includes/defs. I didn't try to integrate or run it yet, just compiled it.

[viciious]

Gonna post this link here for reference purposes: http://fd.fabiensanglard.net/quake3/The-Quake-III-Arena-Bot.pdf

[dnk777]

I have read this paper already. And yes, the ioquake3 AAS code that I use looks monstrous and scary, but I used for overview https://github.com/id-Software/DOOM-3-BFG/blob/master/neo/aas/AASFile.h, they refactored it over the years.

[dnk777]

AAS files are loaded well (I've tried all cpma mappack), atm I just initialize/shut down AAS code and load a map. I have a question, how to implement AAS debug drawing https://github.com/dnk777/qfusion/blob/a5a54ffb5fac083e404be6de45484ec4d197326e/source/game/ai/aas.cpp#L217 (how to check whether I have a client attached and draw via the client, spawning server events is not an option since its very limited)? I need debug visualisation asap, I want to see what the AAS thing is.

[viciious]

The best approximation would be to spawn an ET_LASER entity in AAS_QF_DebugLineCreate, delete it in AAS_QF_DebugLineDelete and assign draw parameters AAS_QF_DebugLineShow

[dnk777]

Quick report: AAS works well, the hardest part is to figure out how to get a chain of AAS areas connected by reachabilities that leads to target, Q3 interface is quite confusing. Introduction of AAS is the most radical change I made, all nav-node-related code has been removed, and it touches everything. Atm the code is in my local repo, I continue to re-implement old code using AAS now.

http://webmshare.com/K7O7q

On the video combat is disabled, and there are known bugs (bot touches items, and when he can't pick them, he gets assigned the item as a goal again since the last goal item is the closest one, pickup code will be improved and on-pickup-actions will be introduced (e.g. damage himself to pick armor)). I had to shoot the bot to make him pick up health, and kill him since he can't pickup RL ammo anymore. Debug drawing is my custom, gold things are goalTargetPoint's in goalAasArea, purple ones are analogs of old self->ai->move_vector, blue ones of various length show AAS reachability vectors.

[viciious]

The navigation towards goals is quite odd, the bot is noticibly circling around them before finally picking them up. I recall a similar problem I observed with bot movement: they tried to move towards the goal while rotating the view and ended up spiraling around just like in your video.

[dnk777]

Yes, its like a movement in a planet gravity field, if bot has a "tangential" velocity, he keeps walking around the target, its a control theory problem. I have already added a workaround (otherwise bot walked around the target forever), its just not tweaked enough for allowed bunny speed. The workaround is checking dot product between actual move dir and a dir from self to target if we are close to target, if its less than a threshold, do one of the following things:

• Roll view harder
• Determine on which side of move dir line the target is. Then release +forward and press strafe to the target side, bot will turn using aircontrol.

[dnk777]

Upd: Basic running movement has been implemented (I did not touch juppads, platforms, etc.). The code is still in my local repo. It it very dodgy atm, since bots tends to stick to nearest AAS reachability, as they used to do it with old nodes. However, it is still a great improvement and a great simplification, we do not need to check whether we have touched a node or not, we may just check AAS area number of current origin point (which is quite cheap, a single BSP lookup).

http://webmshare.com/nrgyr

I plan to smooth actual bot movement path setting current move dir as a weighted sum of dirs to current and some next reachabilities with decay, then add «true» strafejumping (atm it may be implemented but will not be very useful, bots will bump into walls permanently).

I have questions:

• How to implement tracking of allocated/deallocated memory for AAS subsystem. I have to provide real available for AAS free memory amount instead of this stub https://github.com/dnk777/qfusion/blob/aas-port/source/game/ai/aas.cpp#L186, otherwise AAS subsystem never clear cached routes. The problem is how to retrieve size of memory chunk freed in https://github.com/dnk777/qfusion/blob/aas-port/source/game/ai/aas.cpp#L181. Is exposing memheader_s defined in https://github.com/Warsow/qfusion/blob/master/source/qcommon/mem.c#L33 to qcommon.h an acceptable thing?
• How to implement arbitrary debug AAS drawing on a listen server. Using broadcasted events is not an option, since they cause edicts overflow, and I want to be able to draw polys too. The problem is how to add debug drawing to a server code (that may be exported by a game module). May I add something like DrawDebugLine, DrawDebugPoly like, for example, SCR_DrawLoadingPlague in https://github.com/Warsow/qfusion/blob/master/source/qcommon/qcommon.h#L1029` and use stubs for dedicated servers?

[viciious]

• No, you should not include qcommon headers, which are private to the engine core, from game modules. Just get rid of AvailableMemory and HunkAlloc calls altogether, this is ancient stuff. You could also wrap the allocated chunk into helper struct: typedef { int chunk_size; void chunk; } helper_foo; and return pointer to chunk. Then, restore the pointer to helper_foo \ in free.
• The short answer, yes, that would be the best approach. You'll need to proxy-pass your DrawDebug* calls down to cgame, to get your debug stuff rendered properly.

[dnk777]

HunkAlloc is used by AAS subsystem to load AAS data, i backed it by G_LevelMalloc. Having valid AvailableMemory is mandatory, otherwise AAS routing code will not clean cache and will allocate chunks forever: https://github.com/ioquake/ioq3/blob/master/code/botlib/be_aas_route.c#L1624

[viciious]

Well, you can always change those few lines to work differently.. Otherwise, create a separate memory pool for AAS and track its memory usage like I outlined above.

[viciious]

Here's another solution: add the following function to g_utils.c: static int G_Z_AvailableMemory( const memzone_t *zone ) { return zone->size - zone->used; } and use it for G_LevelAvailableMemory and then for AAS_MemoryAvailable.

Use a separate memzone akin to levelpool for AAS hunk allocations. Free it along with the levelpool.

[dnk777]

Recent movement video, looks promising, even GREAT considering how broken bot movement used to be: http://webmshare.com/VN95P. I have not implemented yet neither climbing/jumppad/platform/RJ movement, nor even walljumping, its just generic running. I'm going to push first public version of AAS port in 1-2 weeks.

[adem4ik]

would be nice to see bot's POV video

[viciious]

Nice!

[dnk777]

Upd: I decided to push the AAS code in public access though it is incomplete and poor yet. https://github.com/dnk777/qfusion/tree/master Combat is still disabled and only minimal movement styles set (running/bunnying/jumppad using) has been implemented. And this change was very invasive, I had to reimplement half of all AI code and break script compatibility.

Known bugs: sometimes bots stand still blocked and blocking recovery code is not triggered until suicide, source of the bug has not been spotted yet.

Known issues (not a bugs, need to be tweaked):

• Bots may fail in gaps (i try to shortcut movement path, and there are not cheap ways to check for ground/failure), path straightening/shortcut code is very basic.
• Bots may spin weird, it is an attempt to implement 180-degree turns with dash (when you release all keys in air, then turn mouse very quickly to a target and hit dash on landing), it works but looks robotic.
• Bots use fake acceleration, it is probably bad thing for netcode prediction, but since bots are in development and are tested only on localhost it is not a big issue. Implementing a true strafejumping is not a big deal though.
• The code still leaks (AAS routing cache is never cleaned up), I did not implemented proper AAS allocations handling yet.
• Combat compatibility has not been tested yet.

http://webmshare.com/R5Dq7

Overall the current bot movement is far from being complete, and I do not consider it not good, not even satisfactory. Frankly the code is a one big hack, since AAS provides good support for bots … walking on ground with 320 ups, i just try to fly/bunny over the map and check for turns/obstacles in runtime. I am surprised it even works overall. AAS however is still better than old nav nodes since AAS works with (convex) volumes, not points and greatly simplifies reachability checks.

[dnk777]

Let me describe upcoming changes... I think about 60% of work is done. Release is still deferred to June. Aside from debugging and polishing current code, i plan to introduce multi-level AI brain.

• AiBaseGametypeBrain. This entity should exist as a single instance and think once per frame. It should be used for knowledge exchange among all AI entities present in game regardless of their team and for interaction with gametype, e.g. for cancellation of goals reached by a single bot for all AIs, searching for noobs by rankings to hunt, etc.
• AiBaseTeamBrain. A team is a group of bots (and humans) that share score and strategic goals. It should group bots in squads and determine strategic goals (e.g. schedule powerup pickup, distribute armor pickup between members of a team, etc).
• AiBaseSquadBrain. A squad is a group of bots (and humans) that roam and fight together. Squad brain should suggest tactics: relative and absolute position of bots, prioritize enemies, etc to implement behaviors: «assist carrier», «assist item pickup», «prevent capture», «prevent item pickup», «drop and pickup weapon», etc.
• AiBaseBrain - already exists with its subtype, the BotBrain.

Of course it should be made scriptable, but some basic skeletons for gametype («basic ffa gt brain», «basic TDM-like gt brain», «basic round-based gt brain»), team and squad brain will be provided in AI code.

It is easier to implement that it sounds (it is a pure programming logic, no need to dive in physics simulation, dated Q3 code, fight bot blindness in 3D world, etc).

And of course I am acknowledged about current performance… issues, but won't fix them by distributing workload between frames until the final control flow will be known (obviously team/squad members should think in same or consequent frames).

[Mark90]

Sounds awesome. Looking forward to seeing the result both in code and ingame :)

[viciious]

@dnk777 Any status updates?

[dnk777]

https://github.com/dnk777/qfusion/commits/master.

I fixed some bugs, mainly performance issues by introduction of AiBaseTeamBrain. That class is a stub atm, it only manages think frames distribution between members of a team (which is not its direct purpose but it is required for planned changes), but does not provide any teamplay-related logic. There is a known bug, gametype change is not tracked properly (I do not know the engine control flow well). How may I contact dev team to ask dumb questions instead of spamming in this thread?

Also, this https://github.com/dnk777/qfusion/commit/6537cd5e3ea73169843edc63b734124bfaf1a8db commit fixed critical performance drops (N^2 trace count explosion)

Bots are playable in FFA (not DM).

I think I should defer complex teamplay logic implementation, bots are too weak individually, they can't even pursuit enemies atm. In worst case I think individual bot logic will be improved and polished in these 2 months.

[viciious]

How may I contact dev team to ask dumb questions instead of spamming in this thread?

I think you could instead open new issue(s) for your questions. I'd prefer to keep our communication public, as a sort of documentation process.

[viciious]

I've also started working on qfusion/warsow-compatible version of bspc, should be completed sometime this week.

[viciious]

https://github.com/dnk777/qfusion/blob/master/source/game/ai/aas/be_aas_reach.c#L2468

        ladderface1vertical = abs(DotProduct(plane1->normal, up)) < 0.1;

        ladderface2vertical = abs(DotProduct(plane2->normal, up)) < 0.1;

        //there's only reachability between vertical ladder faces

        if (!ladderface1vertical && !ladderface2vertical) return qfalse;

        //if both vertical ladder faces

        if (ladderface1vertical && ladderface2vertical

                    //and the ladder faces do not make a sharp corner

                    && DotProduct(plane1->normal, plane2->normal) > 0.7

                    //and the shared edge is not too vertical

                    && abs(DotProduct(sharededgevec, up)) < 0.7)

This whole block confuses abs with fabs multiple times

[dnk777]

This whole block confuses abs with fabs multiple times

I think it worth sending a PR to upstream/ioq3, isn't it? And it is not an AAS runtime code, so it does not affect bot runtime behaviour aside from providing some buggy AAS files.

Some notes on AAS:

There are travel flags that correspond to AAS reachabilities: https://github.com/Qfusion/bspc/blob/master/deps/botlib/be_aas.h#L37 I have never see any Q3 maps (including CPMA ones) that contain strafejump, rampjump or doublejump reachabilities. AAS compiler code that determines reachabilies is located in https://github.com/Qfusion/bspc/blob/master/deps/botlib/be_aas_reach.c Strafejump reachability may be added by modifying https://github.com/Qfusion/bspc/blob/master/deps/botlib/be_aas_reach.c#L2103 for increased gap distance. Also, for very large gaps that may be shortcut using horizontal weaponjump (or just very high speed) a new reachability (horizontal weaponjump) may be introduced. Also, you may try to modify jump reachabilities and introduce new walljump reachabilities when target area is higher than current area and a side wall exist. AAS compiler and runtime code are not dependent, so only an introduction of new travel flags may require runtime code patching as well.

The original code searches for reachabilities to a target area only if target area contains an important item, and item names are hardcoded. https://github.com/Qfusion/bspc/blob/master/deps/botlib/be_aas_reach.c#L3938. I think that GB-jump reachability may be introduced, and a teleport entrance is a target that worth weaponjumping too.

AAS files may be optimized(stripped): almost all area faces and edges (aas_face_t, aas_edge_t) are removed, and only coarse AAS area infos (center, AA-bounds and flags) and AAS reachabilities are left. I strongly suggest to avoid stripping for newly compiled maps, AAS geometry data is/would be extremely useful and may replace many runtime traces in bot code which are expensive. For stripped maps (if someone loads Q3 stripped map) we have to fallback to traces, and due to very limited traces count and coarse nature of traces it may cause poor bot behaviour.

[viciious]

I think it worth sending a PR to upstream/ioq3, isn't it?

I will submit the fix to upstream bspc.

Here's FBSP-aware build of bspc which can be used to produce AAS files for Warsow maps in the following fasion:

./bspc -bsp2aas basewsw/map_wbomb6.pk3/maps/wbomb6.bsp -forcesidesvisible

https://github.com/Qfusion/bspc/tree/qfbsp

[viciious]

Finally tried this thing.. I couldn't quite get what the bots were up to all the time with they random movement and missing item pickups. They also seem to enjoy aiming straight up alot. The aas/botlib code will need to be moved to its own library so it doesn't pollute the actual game code.

[viciious]

@dnk777 May I suggest you to resume your development off a different branch? I've managed to rebase your work off qfusion master: https://github.com/Qfusion/qfusion/tree/dnk777_rebased

[dnk777]

I've managed to rebase your work off qfusion master

Ok thanks, got it.

The aas/botlib code will need to be moved to its own library so it doesn't pollute the actual game code.

The AAS code came with its own exports, but I found these exports not useful, since I need access to AAS guts. While I might modify exports, I thought that I should avoid extra work maintaining exports that are changed frequently. I will separate AAS code and restore exports when the final AAS interface required for AI code is known.

Bots have weird behavior

I did not claim that the work is either finished or even is satisfactory. I told planned release date many times. I have just started a work on the bot logic and not on mandatory but auxillary code.

[viciious]

Ye-ye, I know. I'm not criticizing the WIP, just sharing my impressions of this stage of development, as well as pointing out the flaws which otherwise might get lost. I will also try the master branch again later today to see if it's any better than the aas-port branch.