Agents stop running after several steps

[zheyangshi]

Describe the bug All the previous steps worked well and the reward also reached a satisfying value. However, when we started to test and set the Inference Device as GPU, agents would stop moving after several steps (Just stay in the same place). At the same time, we noticed that the GPU was still running when agents stopped.

Screenshots Begining: After several steps:

Environment (please complete the following information):

• OS + version: windows10
• ML-Agents version: Ml-Agents 0.9
• Environment: VisualPushBlock

Parameters: trainer: ppo batch_size: 1024 beta: 1.0e-2 buffer_size: 10240 hidden_units: 128 max_steps: 5.0e5 memory_size: 256 learning_rate: 3.0e-4 lambd: 0.9 normalize: false num_epoch: 3 num_layers: 2 epsilon: 0.2 time_horizon: 64 use_recurrent: true sequence_length: 64 summary_freq: 1000 vis_encode_type: resnet pretraining: demo_path: ./UnitySDK/Assets/Demonstrations/VisualPushBlock.demo strength: 0.5 steps: 10000 reward_signals: extrinsic: strength: 1.0 gamma: 0.99 curiosity: strength: 0.02 gamma: 0.99 encoding_size: 256 gail: strength: 0.1 gamma: 0.99 encoding_size: 128 demo_path: ./UnitySDK/Assets/Demonstrations/VisualPushBlock.demo use_vail: true

[ervteng]

Hi @zheyangshi, do you mean GPU in the Brain settings, and inference with CPU works?

This could be a number of issues - I haven't been able to replicate on my end. What Unity version and GPU are you running? Thanks!

[zheyangshi]

Hi @ervteng, Here is the picture of our brain setting. We tried on the following environments, but we still meet the same problem. Unity version: 2018.2.17/2017.4.17f1 GPU: GTX 1080/ GTX 1070

What's more, we found that the value of act[0] would become 0 after several steps and that's why the agents stayed in the same place. (the accident usually happens when tasks are being finished.)

[arixlin]

@ervteng I has same errors, when I using vsiual observation and GPU in the Brain settings run on internal, but Console dont print any errors. Thanks! VisualPyramidsLearning.nn file: https://drive.google.com/file/d/1FmT8Ysw8CkW4dA1cY-KTkSXB84L1t-YC/view?usp=sharing

[ervteng]

Hey @zheyangshi, @arixlin, just to let you know we're looking into this. Just to confirm, CPU inference works fine, right?

[zheyangshi]

@ervteng, Thank you very much. CPU inference works fine. The issue just happens in GPU inference.

[arixlin]

@ervteng Right, CPU inference works fine, but it is slow. When I train multiple envs at the same time using tensorflow-gpu==1.7.1（--num-envs=5 --slow）, there will be some agents to stop acting and others is fine, Thanks!

[ervteng]

@arixlin I guess I was asking about CPU inference from inside Unity, not Python - the Barracuda setting shouldn't affect training at all.

[ervteng]

@arixlin, just wanted to elaborate on my prior answer. The toggle inside the Unity editor changes the inference mode of the in-game Unity Inference Engine (Barracuda). It shouldn't be affected by Tensorflow at all.

Tensorflow-GPU not working is another issue, though. In my experience tf-gpu works fine with multiple environments. Do the agents still get stuck when you're training with one env?

[arixlin]

@ervteng Yes, some of them will get stuck but no all when i using tensorflow-gpu. I guess the situation is not caused by the tensorlfow-gpu or Barracuda setting, because both internal and external occur, Thanks your reply.

[harperj]

Thanks for the discussion. We are closing this issue due to inactivity. Feel free to reopen it if you’d like to continue the discussion.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.