Implement Smart AI Players

[zicklag]

With #272 we re-implemented extremely basic AI, but we want it to be much smarter than that long term.

This issue is to get an actually semi-intelligent AI that will play as close as normal player as we can get it.

Topics of investigation include reinforcement learning and behavior trees.

[zicklag]

Something I've been looking into that might be useful to try is OpenNARS for Applications.

It's written in C which is a disadvantage, but it communicates over stdin/stdout so it wouldn't be hard to integrate with.

WASM would be a question, but we should be able to compile it to WASM with emscripten and then talk to it over WASM bindgen. The communication is literally just lines of strings in and lines of strings out.

If we got a reinforcement learning agent that worked well and was written in Rust that'd probably be preferred, though, unless NARS created a much better AI somehow.

[erlend-sh]

The more we talk about different approaches to Bot AIs, the more it feels like we might as well adopt a free-for-all approach to bot implementations.

One would be the default, but expert users would know they could toggle between various different AI systems in the game settings.

[zicklag]

Yeah, we might as well let players mess with which AI they want to play against.

We should be able to make the AI a WASM script eventually, too, so mods could make new AI engines.

[zicklag]

Just an update on OpenNARS for Applications, it's not really in a good state for using. I created Rust bindings, but quickly ran into limitations.

So far the best option I see for implementing this is to use reinforcement learning powered by the native Rust dfdx crate.

[zicklag]

There is an additional possible direction for AI strategy called "Active Inference". In my initial research it does seem superior in concept to reinforcement learning, but it is less widely used.

More research is necessary.

Some possible research resources include:

• https://github.com/pl-robotdecision/tutorials-active-inference/blob/main/exercises/1_FEP_TheSensingWheel_Perception_exercise.ipynb
• https://msrmblog.github.io/is-this-my-body-1/
• https://msrmblog.github.io/is-this-my-body-2/
• https://github.com/BerenMillidge/FEP_Active_Inference_Papers

[MaxCWhitehead]

One of the issues having with AI now that have round scoring (players don't respawn on death) is that sometimes the AI will keep attacking corpses or get stuck and the round never completes.