Looking for Accord.NET game ideas and use-cases

[FireDragonGameStudio]

What would you like to submit? (put an 'x' inside the bracket that applies)

• [x] question
• [ ] bug report
• [ ] feature request

Issue description Hy, as I'm currently looking for ideas, how to apply the functionality of Accord.NET to Unity3D games, I was wondering, if there are any ideas floating around here. I'm not looking for source code or already existing solutions, just ideas and use cases for creating prototypes. I think it goes without saying, that I'll share everything here, to extend the sample section with useful information and projects. :)

Maybe I'll just start with a few examples, that came to my mind:

• Using the gesture controller for a Panzer Dragoon like mobile game, where the aiming is done with the help of head- and/or facecontroller. As nowadays every smartphone has a front camera, this could be a really convenient way of user input.
• Using the gesture recognition for creating a gesture database for an Okami-like mobile game, using custom gestures (as they are always needed in mobile games) for different player actions (as drawing symbols was used in Okamit to trigger special powers).

Best regards :)

[cesarsouza]

Hi @FireDragonGameStudio!

Thanks for opening this very interesting issue! A few weeks ago I was also trying to look for ideas regarding Unity games that could have been implemented with the help of Accord.NET since I was trying to come up with a sample application for Unity.

Currently the framework doesn't offer many implementations that could have been directly applicable to games at this point. However, I guess that some sample applications, such as the Animat would have been easy to be adapted for Unity - all that would have been needed would have been a labyrinth-like scene where the agents would need to navigate the 3D environment on their own towards a particular target.

Regards, Cesar

[FireDragonGameStudio]

Hy @cesarsouza, thank you for your suggestion. The Animat example was easy to adapt, although a few smaller problems occured, regarding the correct integration of everything into the Unity gameloop. I'll rebuild the Animat sample application in Unity and as soon as I'm finished, I'll send you the sampe project, so you can integrate it into the part of the Unity sample applications. :)

Best regards :)

[cesarsouza]

Amazing! Thanks a lot, that would be a very welcome addition!

[cesarsouza]

Hi @FireDragonGameStudio,

Well actually I might have some more ideas related to Unity, but not necessarily related to game mechanics. I think it would be really exciting to see a sample that could play a video game automatically. I was thinking about the feasibility of gathering an example FPS Unity demo (let's say, a random FPS game engine in the store such as this one), and writing an AI that could at the very least navigate their scenarios. This could still be done using Q-Learning, but it would need to be done in the same way as the original WinForms application where the current state would be based on local information about the environment around the "virtual" player.

Additionally, it might be possible to make it also follow a target using something like the Camshift tracker (this is a very simple tracker, but its the only one available at the moment - better ones might be added in the future). It would have also been possible to identify targets on screen using the Haar object detector (but unfortunately the detectors would have to be trained outside of the framework using OpenCV, at least until proper Haar training gets eventually added to the framework).

Otherwise, another idea would be to keep track of many game variables and determine the likelihood of some event happening. For example, by gathering many examples of players playing a particular game, it would be possible to create a dataset of player behaviors and match outcomes. This would make it possible to establish a mapping particular user behaviors to whether this player has won or lost a match, and therefore create an AI that can predict the outcome of a game.

Regards, Cesar

[FireDragonGameStudio]

Hy @cesarsouza, thank you for the great ideas and suggestions. It's a very funny coincidence, because my master thesis (which I'm currently working on) covers these topics (or at least parts of it ^^). I already implemented a navigation (with random movement, for arena fights) AI player with the help of DeepBeliefNetwork, which will be modified and adapted for QLearning and maybe some kind of DQN.

Object detection with images is also a very insteresting way of getting input data, but I skipped this method and replaced it with new (self-implemented) AI sensors for vision and audio. As this features are still under development, I'll provide additional information, as soon as my thesis is finished and everything was tested.

The prototype I'm working on right now, is quite similar to your last suggestion. It is actually simpler, as the AI will track player behaviour and trigger events. E.g. a stealth game and every time the player is spotted, the AI will try to predict where the player is heading to and send reinforcements to that goal. Fuzzy logic/classifiers will be used for this.

As soon as I'm finished, I'll provide you with the sample applications I came up with. :)

Best regards :)

PS: I attached you a short demonstration of the navigating AI with DeepBeliefNetwork that I mentioned before. It's not working perfect, but I think it will give a good impression, of what can be achieved. :)

DeepBeliefNetwork AI sample

• Green: reward (like energy etc...)
• Red: punishment (like enemy, trap, etc...)
• Blue: player
• Rays: FoV (for this sample, the proximity of vision is very short, like a flashlight, referring to a dark environment)