Cycloid-path walking behavior in NERO mod

[GoogleCodeExporter]

I know you guys are already aware of this problem, but I wanted to start this 
ticket as a place to attach ideas for fixes. Apologies if this is not the 
appropriate location for this discussion.

I am currently running revision 1264 built on 64-bit Arch linux.

Place a flag on the field, and train the robots to approach the flag.

Even after they learn how to approach the flag, they won't walk straight there 
(as they do in torque NERO). Instead they follow a cycloid or spiral path in 
the direction of the flag.

Possible solutions:
1) I discussed this with Risto at class last night; we were wondering if 
weighting the fitness with the amount of time the robots spend near the goal 
instead of simply how close their final position is might incentivize them to 
follow a more direct path.

2) It also seems suspicious that successful agents are the ones who wind up 
circling the target. If they have learned that circling is awesome behaviour, 
wouldn't that influence their paths all the time? Risto said that one thing the 
torque code does is "cheat" the agents to always face enemies that are in their 
field of view. It seems like this kind of tweak might prevent unnecessary 
circling.

Original issue reported on code.google.com by jroes...@gmail.com on 2 Nov 2011 at 11:16

[GoogleCodeExporter]

Thanks for the issue. A couple of comments below.

> 1) I discussed this with Risto at class last night; we were wondering if 
weighting the fitness with the amount of time the robots spend near the goal 
instead of simply how close their final position is might incentivize them to 
follow a more direct path.

Their reward is currently based on cumulative distance from flag over the 
course of their lifetime. Thus, agents that, on average, spend more time closer 
to the goal should get higher raw fitness value.

> 2) It also seems suspicious that successful agents are the ones who wind up 
circling the target. If they have learned that circling is awesome behavior, 
wouldn't that influence their paths all the time? Risto said that one thing the 
torque code does is "cheat" the agents to always face enemies that are in their 
field of view. It seems like this kind of tweak might prevent unnecessary 
circling.

In Torque-NERO this was an unintentional side effect of the auto-aiming feature 
- even if the agent is signaling a turn, having an enemy in its sights will 
cause it to face the enemy as long as its firing output is on. In effect this 
makes it easier to approach enemies (sometimes too easy, as it is hard to teach 
Torque-NERO agents to NOT approach the enemy).

Original comment by ikarpov on 2 Nov 2011 at 3:11

• Changed state: Accepted

[GoogleCodeExporter]

scaling issues addressed on trunk revision 1365 should at least partially 
address this issue.

Original comment by ikarpov on 11 Nov 2011 at 9:42

• Changed state: Fixed