Closed dhruvmsheth closed 1 week ago
Hello! This question would have been a good fit for the MuJoCo repo as well, as it concerns the plugin. A couple issues at first look:
Thanks a lot for the suggestions @Balint-H! I made the updates and tried the script again but the frames still seem to be choppy. The suggestions were useful though, I completely missed them.
As far as performance goes, I don't see anything in your script that I'd expect to be cause of slowdown. The only other thing I can think of, is how many dofs/instances do you have in your scene? I do quite heavy computations each frame in my scene (heavily unoptimized), and I can only have up to 5-6 humanoids while maintaining good framerates. If you are trying dozens then the plugin might struggle with that.
You could use the Unity profiler to identify your bottleneck.
Other minor considerations:
Academy.AutomaticSteppingEnabled
to false in the init, then subscribe Academy.EnvironmentStep()
in the MjScene.Instance.ctrlCallback
to not have to worry about the MjScene being initialized in the Action and Observations.Do you get any exceptions or error messages in the editor? Also, if you have termination conditions, check whether they are triggering. Its possible they are continuously resetting your scene.
Thanks for the prompt reply @Balint-H! My model is sufficiently lightweight and only has 4 joint with 1 free joint and there exists only 1 instance of it. I'll definitely try using profiler and let you know if I find something. The only exception I have in my terminal is "Couldn't connect to trainer on port 5004 using API version 1.5.0. Will perform inference instead."
. I don't think I have any termination conditions, I manually terminate it using the python mlagents api. The interesting thing is that when I control the simulation locally using the mujoco plugin, it works perfectly fine and the physics is decent as well with high FPS. However, the problem starts coming in when I attach the MLAgents script to the agent. Even before I build and control the agent using the python api, just when I attach the script and hit play, the simulation becomes choppy and goes to 1FPS. That makes me suspect it's an initialization issue in MLAgents and I'm probably just missing something or maybe there's some interference between the script that locally controls it and the MLAgents script. I don't attach the script that locally controls it when MLAgents script is attached to the simulation. Additionally, when I change the decision period to 20, the simulation has a slightly higher FPS but the choppiness still exists. When the decision period is 1, the FPS is less than 1. Thanks for all the help!
Here's the script I use to locally control it:
That type of scene should be able to easily reach a few hundred FPS even with ML Agents. Sorry that I can't take a more in depth look right now, but using the profiler, and trying to isolate what causes it (e.g. by commenting out the call to the getState function, or the call to differentiation API) could help. Also, if you are looking for examples of Mj + ML Agents, you can take a look at how scenes are constructed in ModularAgents, a repo of mine that has components, extensions and scenes for this kind of setup.
Thanks for the help! I'll try that out and let you know
This issue is stale because it has been open for 30 days with no activity.
I've created a simple MLAgents script that sends the state information of the joints of the object and retrieves the velocity information for the actuators as input so that the joints can move to the desired state. I'm not using the MLAgents training procedure here, I'm using MLAgents in unity simply to interface with the python low level api, receive the state input, process it using a PIDcontroller and send the actuations for joints back to execute. When I create a C# script to execute this process natively on unity without MLAgents, the simulation is extremely smooth (My timestep is 0.02) and the motion is continuous. However, when I create the MLAgents C# script and remove the previous script and simply hit on play, the simulation becomes extremely choppy and starts executing the same process at 1FPS. My decision requester is set at 1 simply because I'm not training a model and I want the PIDController to retrieve the state information at all frames to be processed and imitate the mujoco simulation I had. I'm new to MLAgents and for that reason I might be making some obviously dumb mistake but any help would be appreciated. Sorry for tagging you @Balint-H but since you have worked with Mujoco + MLAgents, I thought tagging you might help.
Environment (please complete the following information):
Attaching the script and the MLAgents configuration here if that helps.
MLAgents script that causes the simulation to become choppy:
```csharp using System.Collections; using UnityEngine; using Unity.MLAgents; using Mujoco; using Unity.MLAgents.Sensors; using Unity.MLAgents.Actuators; using System; using System.Linq; using System.Collections.Generic; public class agentCallback : Agent { private double[] initialState; private double[] currentState; private MjScene mjScene; private bool isInitialized = false; private PIDControl pidController; private int nSteps = 0; public double targetHeight = 0.1; public double targetRadius; public double targetAngle; public override void Initialize() { base.Initialize(); StartCoroutine(InitializeMjScene()); } private IEnumerator InitializeMjScene() { while (MjScene.Instance == null) { yield return null; } mjScene = MjScene.Instance; isInitialized = true; } unsafe public override void OnEpisodeBegin() { // base.OnEpisodeBegin(); if (!isInitialized) { Debug.LogWarning("MjScene not initialized yet. Waiting..."); return; } MujocoLib.mjModel_* Model = mjScene.Model; MujocoLib.mjData_* Data = mjScene.Data; MujocoLib.mj_resetData(Model, Data); int stateSize = Model->nq - 4 + 3 + Model->nv + Model->na; initialState = getStateInformation(mjScene, stateSize); currentState = new double[stateSize]; Array.Copy(initialState, currentState, stateSize); pidController = new PIDControl(targetAngle, targetRadius, targetHeight); } unsafe public double[] getStateInformation(MjScene mjScene, int stateSize) { MujocoLib.mjModel_* Model = mjScene.Model; MujocoLib.mjData_* Data = mjScene.Data; double[] dq = new double[Model->nv]; fixed (double *dqPtr = dq) { MujocoLib.mj_differentiatePos(Model, dqPtr, 1, Model->qpos0, Data->qpos); } stateSize = Model->nq - 4 + 3 + Model->nv + Model->na; double[] currState = new double[stateSize]; int index = 0; ListMLAgents configuration details:
Thank you for the help!