StepCount increasing faster when training

[22ms]

Hello! I have been getting into Unity ML-Agents lately and been loving it so far! There is one thing that has been bugging me tho. I can't figure out why, when training, the StepCount increases significantly faster than when doing heuristic or inference. I've set the timescale to 1 but still, it increases like 2-3x faster. The same goes for trying to Invoke methods or waiting in coroutines, the time scale seems to be off even tho it's at 1. Do you know why it is like that or how to "fix" this issue? I want to have timed events the AI has to react to, is there a better way to do so? Appreciate any help!

[ervteng]

Hey @22ms - this seems worth looking into. Have you been able to reproduce with one of the example environments?

[22ms]

Yes, I can reproduce it in any environment. I use Python 3.7.9 and have everything properly installed. No errors or warnings what so ever. When I set the BehaviourType to Default and train the AI it does train how it should and sped up. But when I set the time scale to 1 (--time-scale 1) the Time.timeScale in unity is in fact set to 1, but the step count increases faster thus the game is faster than heuristic or inference and not at all time scale 1 but more like 2 or 3. Using a little script I wrote I evaluated how many steps happen in a second. When doing heuristic or inference, it is at 50 which should be time scale 1. When training at time scale 1, it is at about 150. Here is the script I wrote:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using Unity.MLAgents;

public class GameSpeed : MonoBehaviour
{
    public Agent agent;

    void Start()
    {
        StartCoroutine(OneSec());
    }

    private IEnumerator OneSec()
    {
        int stepCountStart = agent.StepCount;
        yield return new WaitForSecondsRealtime(1);
        Debug.Log(agent.StepCount - stepCountStart);
        StartCoroutine(OneSec());
    }
}

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 14 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 28 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.