NullReferenceException in Agent.cs after upgrades-downgrades

[AMGMNPLK]

Describe the bug I get null references around agent in a custom environment that have been working in past versions since the very beginning of ML-Agents (v 0.3). The last functional version was Release 1, the same that now doesn't work. I tried to upgrade to join other scenes to the project (not to the env), and I downgraded it after seeing too many errors with ML-Agents Release 7.

To Reproduce Steps to reproduce the behavior:

1. Run training in console (also in the editor)
2. See errors

Console logs / stack traces First error:

NullReferenceException: Object reference not set to an instance of an object Unity.MLAgents.Agent.LazyInitialize () (at com.unity.ml-agents/Runtime/Agent.cs:395) Unity.MLAgents.Agent.OnEnable () (at com.unity.ml-agents/Runtime/Agent.cs:303)

The second error in a loop:

NullReferenceException: Object reference not set to an instance of an object Unity.MLAgents.Agent.ResetData () (at com.unity.ml-agents/Runtime/Agent.cs:726) Unity.MLAgents.Agent._AgentReset () (at com.unity.ml-agents/Runtime/Agent.cs:1112) Unity.MLAgents.Academy.ForcedFullReset () (at com.unity.ml-agents/Runtime/Academy.cs:480) Unity.MLAgents.Academy.EnvironmentStep () (at com.unity.ml-agents/Runtime/Academy.cs:492) Unity.MLAgents.AcademyFixedUpdateStepper.FixedUpdate () (at com.unity.ml-agents/Runtime/Academy.cs:34)

Player logs:

NullReferenceException: Object reference not set to an instance of an object at Unity.MLAgents.Agent.LazyInitialize () [0x000c5] in C:\folder\com.unity.ml-agents\Runtime\Agent.cs:395 at Unity.MLAgents.Agent.OnEnable () [0x00001] in C:\folder\com.unity.ml-agents\Runtime\Agent.cs:303 UnityEngine.Object:Internal_CloneSingleWithParent(Object, Transform, Boolean) UnityEngine.Object:Instantiate(Object, Transform, Boolean) UnityEngine.Object:Instantiate(GameObject, Transform, Boolean) UnityEngine.Object:Instantiate(GameObject, Transform) CustomEnvLoader:GetElementFromPool(List`1, GameObject) (at C:\folder\CustomEnvLoader.cs:135) CustomEnvLoader:JubotsInstantiation() (at C:\folder\CustomEnvLoader.cs:298) CustomEnvLoader:CustomEnvInstantiation(CustomEnv) (at C:\folder\CustomEnvLoader.cs:96) CustomEnvController:ResetTraining(CustomEnv) (at C:\folder\CustomEnvController.cs:231) AuxiliarAcademy:EnvironmentReset() (at C:\folder\AuxiliarAcademy.cs:94) Unity.MLAgents.Academy:EnvironmentReset() (at C:\Users\alvar\RiderProjects\Representador-Dev\com.unity.ml-agents\Runtime\Academy.cs:530) Unity.MLAgents.Academy:ForcedFullReset() (at C:\Users\alvar\RiderProjects\Representador-Dev\com.unity.ml-agents\Runtime\Academy.cs:479) Unity.MLAgents.Academy:EnvironmentStep() (at C:\Users\alvar\RiderProjects\Representador-Dev\com.unity.ml-agents\Runtime\Academy.cs:492) Unity.MLAgents.AcademyFixedUpdateStepper:FixedUpdate() (at C:\Users\alvar\RiderProjects\Representador-Dev\com.unity.ml-agents\Runtime\Academy.cs:34)

Environment (please complete the following information):

• Unity Version: Unity 2019.4.10f1
• OS + version: Windows 10
• ML-Agents version: ML-Agents release 1 -- Version information: ml-agents: 0.16.0 (0.20.0, 0.18.0 that is what we had before doesn't work either) ml-agents-envs: 0.16.0 (0.20.0, 0.18.0 that is what we had before doesn't work either) Communicator API: 1.0.0
• TensorFlow version: 2.2.0
• Environment: Custom, it has been working in so many past versions

NOTE: We are unable to help reproduce bugs with custom environments. Please attempt to reproduce your issue with one of the example environments, or provide a minimal patch to one of the environments needed to reproduce the issue.

You can check the Player logs for more clues. I have been debugging all of our references and no one is null.

[andrewcoh]

Hi @AMGMNPLK

Just to make sure I understand what's going on, you have no issues with release 0.3 to release 1 but failure when you upgrade to release 7?

Also, if you downgrade ML-Agents in the same env, these issues go away?

[AMGMNPLK]

Hi @andrewcoh ,

We have been upgrading but skipping some versions. We could make it run, but we had issues sometimes. We also upgraded Unity, and that brought more problems.

When we upgraded to release 7 from release 1, yes, we found failures, but we are not sure from where they came from. We came back soon to release 1. I can't say if those failures went away because we where merging scenes from different projects and we had errors.

Now, with the same code but in Unity3D 2019 instead of Unity3D 2018, we are also having NullReferenceExceptions where we haven't before in other scenes independent of ML-Agents. I don't know if this could bring any clue.

[andrewcoh]

Ah, so you are seeing NullReference issues in other scenes that have nothing to do with ML-Agents when you upgrade Unity?

[AMGMNPLK]

Yes, but the NullReferences that I brought here are inside com.unity.ml-agents, about Agent... as you can see in the logs. Our environment and the agents, academy... worked in other versions of ML-Agents. I followed the migration instruction each time.

Today I tried between different Python versions (0.16.0, 0.18.0...) and the camera doesn't work when I downgrade 🤷 .

[AMGMNPLK]

I think it could be related with the python -m mlagents.trainers.upgrade_config -h. How can I undo those changes?

Another topic, is the Versioning.md updated in each release? because in Release 1, 4, 5... branches have the same versions that file.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 28 days. It will be closed in the next 14 days if no further activity occurs. Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in the last 42 days. If this issue is still valid, please ping a maintainer. Thank you for your contributions.

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.