Jolt Sphere Arena Machine Learning System

[Youseflapod]

This is an issue with the main focus of it being a large discussion on how the machine learning system for Jolt Sphere Arena will function.

[Aspenup]

To begin with, we need to know what specific tasks the ai will complete to be wrapped into one "brain" to ensure successful functioning human player simulator. I think it would be easier to think of it as a brain. We also need to know how to create a synthetic mind that can store memory but not copy it exactly. CPUs in the past do not follow this deep learning method. Instead, they make full on calculations to create the perfect senario for their victory. I don't know how difficult or easy it would be to do that on Jolt Sphere vs a game like chess where there are more limitations on the players. The main reason we do not want a full on calculator is because it would be too difficult. We also need to look at the in game options for the ai. That includes multiple difficulties, an ai with a clean memory within each copy of the game, or a universal ai that will be cut off at a certain point in its learning, to make it easier to beat. I think that there are too many better possibilites than simply cutting off the ai's learning if we are going to use deep learning, I'll get to that on my next post. Anyway, we need a check list for the aspects of the ai, I just named a few.

[Aspenup]

Check list for now

1. How it learns, humans playing against it vs another ai
2. In game options
3. Advantages of deep learning technique that could be utilized to make the game and possibilites more interesting 4.Multiple neural networks that connect to make a functioning "brain"

You guys can add on, these are just a few

[Aspenup]

5. Also, the actual programming of the ai (Yousef, I know you have that deep learning book, I hope it helps out enough)

[Aspenup]

The programming will definitely go into subcategories.

Sorry I posting stuff all over the place, I need to clean up my thoughts

[Youseflapod]

I think we should run the AI on the GPU since the CPU is already being used by the Physics engine

[Aspenup]

Regarding #3 on the list, we should analyze the advantages of using deep learning AI vs a common computer opponent. To prevent an AI from becoming OP, it was suggested that somewhere along the AI's learning/training process, it should be cut off from training any further, but the problem with this is that because it will only have had the memories of the opponent(s) it has faced in the past, it would be unable to utilize different strategies against opponents it has never played. This poses a problem as it will become predictable, and the whole advantage of using an AI rather than a normal computer opponent would be diminished. The ideal AI for this game is not an AI that imitates players in general, but one that imitates the most skilled players. An AI that does not mindlessly click buttons /spam or use one strategy, but one that will find the weak points of any strategy and use a strategy to defeat it. If the player wants to defeat the AI, they must concoct a strategy to defeat the one that the AI is using, and the cycle continues, making a kind of cyclical imbalance. A successful AI simulates the mind of a life form, but in order to do that, it must improve, as all life forms do to survive. A life form learns, either individually through knowledge it gains in life, natural selection, or otherwise. It needs that advantage to survive, or it will become extinct. If we want the game to survive, we need to focus on its unique advantage, and that is the Deep Learning portion of the AI. If we want the AI to survive, we need to focus on its advantage, and that is its ability to learn and improve. All I'm saying is, if there is another option to cutting the AI's learning off at some point to prevent it from becoming OP, we should seize the option.

[Aspenup]

If we do choose to not cut off the learning process of the AI, here are a couple of options that I propose: 1.Using a deep memory AI with its own memory for each copy of the game (that means a completely restarted memory per copy) Advantages: It does solve the problem from earlier and it works as a tutorial at the beginning for both the player and the AI Disadvantages: It's quite similar to Ammibos in Smash Bros though the way it behaves may be different and it won't learn as quickly as the player at the beginning, so it might take a while to catch up. It also will not be as flexible if other strategies are used against it, i.e. Another player player uses it in the same copy of the game

2. A universal AI that learns from every player and is uncut, which means it's constantly changing Advantages: Solves that one problem I mentioned earlier as well as knowing how to deal with Disadvantages: It will develop too quickly and only be enjoyed by the best players because they raised the bar so high. It will be too OP for anybody at some point

That's all I have for now. Feel free to think up of any of your own options. I prefer my first option over the second one for now, but that's only if it's possible (there may be issues regarding an individual AI per copy). We can also have a combination of AI options, including also adding in multiple difficulties and cutting the learning off at those points, etc.

[Aspenup]

Actually, having a combo of AI options seems like an even better option. If we use option 1 that I proposed (which allows for revealing the advantages of using Deep Learning), we can cover its weak spots with the original idea, which was to train it and cut off its learning at some point. To add even more variation to the original idea, we could add multiple difficulties where we have multiple copies of the AI where the learning is cut off at different points. I don't know if that's possible though.

[Aspenup]

We need to come up with a decision of how the AI will be trained before we can come to a conclusion regarding #3 on the list because we still are not sure how the AI will play in different situations. Our options include training it with humans or AI. We will have to test out both to find what we like better but here are some ideas for training the AI with other AI. My next post will be about using people instead.

• One AI training another AI The AI that we will eventually use in the game, lets call it A, will train another AI, B. They will learn from each other and continually use strategies to defeat one another to the point where A would be a worthy opponent for real players. Since it might take a while for the AI’s to get to that point, we could somehow find a way to speed up the game itself, so that the computer runs faster itself and it isn't necessary to watch every game in order for the AI’s to develop. The games could just run on their own and we could drop by and check their process every once in a while. If we don't like where it's going, if their playing is inhuman or a technique is ineffective for example, we could easily reprogram the AI’s to prevent the situation from going further. Through trial and error, a satisfactory result will occur at some point, hopefully.

• Multiple AI’s vs One Yousef’s idea is to have the original AI play against a trainer AI for a while, and then switch to a new trainer so that the original AI will learn to understand how to execute and defend against many playing styles. Because the original AI will learn from the ones before it, the next trainer AI will be forced to develop new styles and the original will learn these new styles as well, until there doesn't seem to be much the original can learn from that training AI. It will then head on to the next. The only problem with this method is that each new trainer AI will take a while to develop to the level of the original, so that they can have a challenging battle and with each new trainer, it will be more difficult for both AI’s to learn much because the original will become more and more powerful, not giving much of a chance to the new trainer in the process. This will make each encounter with a new trainer longer than the one before.

 How to Speed Up the Training Process

• The AI could have a memory of past attacks and eliminate the techniques that are ineffective, and then restart with different attacks, trial and error style • Attack and defense should be within the same priority level so that the two AI’s don't end up fighting for two long, although if that did happen, I guess it wouldn't change much, since the only difference in starting a new game vs being on the same one is how easily one player can knock off another. A separate neural network that drops the chance of using a certain style if it has been largely ineffective in the past Picking out games that the AI has won and using the same moves that it has used in those games. The problem with this one though is that good moves will be lost and only a limited number will be used/it won't be as flexible if it starts losing nonstop against an opponent. It won't be able to adapt unless we restrain some aspects of the idea, though it would be a lot easier to implement.