Unity-Technologies / ml-agents

The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.
https://unity.com/products/machine-learning-agents
Other
17.19k stars 4.16k forks source link

Small error in game Logic - Tennis #670

Closed Rfrixy closed 6 years ago

Rfrixy commented 6 years ago

Rewards for the agents are decided by a hitWall script on the ball. This script has one error- (The following code are from lines 45 to 74 from above script)

            if (collision.gameObject.name == "wallA")
            {
                if (lastAgentHit == 0)
                {
                    agentA.AddReward( -0.01f);
                    agentB.SetReward(0);
                    agentB.score += 1;
                }
                else
                {
                    agentA.SetReward(0);
                    agentB.AddReward(-0.01f);
                    agentA.score += 1;
                }
            }
            else if (collision.gameObject.name == "wallB")
            {
                if (lastAgentHit == 0)
                {
                    agentA.AddReward( -0.01f);
                    agentB.SetReward(0);
                    agentB.score += 1;
                }
                else
                {
                    agentA.SetReward(0);
                    agentB.AddReward( -0.01f);
                    agentA.score += 1;
                }
            }

What's happening here is that if the lastAgent was 0 which means agent was A, and it hits wall A, which is the wall behind agent A's racquet, he gets a penalty and B scores. And also, if B hits A's wall directly, he gets the point.

However, in this part,

            else if (collision.gameObject.name == "wallB")
            {
                if (lastAgentHit == 0)
                {
                    agentA.AddReward( -0.01f);
                    agentB.SetReward(0);
                    agentB.score += 1;
                }

It again checks for agent A's hit and penalizes it for hitting B's Wall.

I propose changing

                if (lastAgentHit == 0)

to

                if (lastAgentHit == 1}

(in the first if statement in the block where it checks for wallB. ) This should resolve the issue.

Rfrixy commented 6 years ago

May I work on this issue and contribute?

awjuliani commented 6 years ago

Hi @Rfrixy ,

Thanks for pointing this out. Please feel free to make a PR to develop branch with a fix for this, if you happen to put one together. It would be greatly appreciated.

Rfrixy commented 6 years ago

I seemed to have made a mistake while dealing with the ball-wall logic, the current code is perfectly fine. I apologize for wasting your time

lock[bot] commented 4 years ago

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.