search

Unity-Technologies / barracuda-release

Other
567 stars 77 forks source link

NotSupportedException: Format version not supported: 244 #254

Closed KamilMatejuk closed 2 years ago

KamilMatejuk commented 2 years ago

Hi. I'm using Unity ML-Agent and successfully trained model, however when I try to load this model via script it returns an error.

Versions: Model format: ONNX OS: Ubuntu 20.04 ml-agents: 0.28.0, ml-agents-envs: 0.28.0, Communicator API: 1.5.0, PyTorch: 1.8.1+cu102, Connected to Unity environment with package version 2.0.1 and communication version 1.5.0

What is interesting is when I physically move model into model field in editor, it works. Only when I try to load it via script it crashes.

The path of model is Assets/Resources/brain.onnx

When loading with method from here:

public static NNModel LoadModel1(string path) {
    byte[] model = null;
    try {
        model = File.ReadAllBytes(path);
    } catch (IOException) {
        Debug.Log($"Couldn't load file {path}");
        return null;
    }
    NNModel asset = ScriptableObject.CreateInstance<NNModel>();
    asset.modelData = ScriptableObject.CreateInstance<NNModelData>();
    asset.modelData.Value = model;
    string[] pathArr = path.Split('/');
    string name = pathArr[pathArr.Length - 1];
    asset.name = name.Split('.')[0];
    return asset;
}

it returns error:

NotSupportedException: Format version not supported: 244
Unity.Barracuda.ModelLoader.Load (System.IO.BinaryReader fileReader, System.Boolean verbose, System.Boolean applyPatching, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/Core/ModelLoader.cs:111)
Unity.Barracuda.ModelLoader.Load (System.Byte[] stream, System.Boolean verbose, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/Core/ModelLoader.cs:68)
Unity.Barracuda.ModelLoader.Load (Unity.Barracuda.NNModel model, System.Boolean verbose, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.0.0/Barracuda/Runtime/Core/ModelLoader.cs:29)
Unity.MLAgents.Inference.ModelRunner..ctor (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.Int32 seed) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Inference/ModelRunner.cs:72)
Unity.MLAgents.Academy.GetOrCreateModelRunner (Unity.Barracuda.NNModel model, Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Policies.InferenceDevice inferenceDevice) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Academy.cs:620)
Unity.MLAgents.Policies.BarracudaPolicy..ctor (Unity.MLAgents.Actuators.ActionSpec actionSpec, System.Collections.Generic.IList`1[T] actuators, Unity.Barracuda.NNModel model, Unity.MLAgents.Policies.InferenceDevice inferenceDevice, System.String behaviorName) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Policies/BarracudaPolicy.cs:84)
Unity.MLAgents.Policies.BehaviorParameters.GeneratePolicy (Unity.MLAgents.Actuators.ActionSpec actionSpec, Unity.MLAgents.Actuators.ActuatorManager actuatorManager) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Policies/BehaviorParameters.cs:240)
Unity.MLAgents.Agent.ReloadPolicy () (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Agent.cs:651)
Unity.MLAgents.Policies.BehaviorParameters.UpdateAgentPolicy () (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Policies/BehaviorParameters.cs:276)
Unity.MLAgents.Policies.BehaviorParameters.set_Model (Unity.Barracuda.NNModel value) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Policies/BehaviorParameters.cs:108)
Unity.MLAgents.Agent.SetModel (System.String behaviorName, Unity.Barracuda.NNModel model, Unity.MLAgents.Policies.InferenceDevice inferenceDevice) (at Library/PackageCache/com.unity.ml-agents@2.0.1/Runtime/Agent.cs:636)
CarAgent.Start () (at Assets/Player/CarAgent.cs:63)

I tried using method from here as well:

public static Model LoadModel2(string path) {
    return ModelLoader.Load((NNModel)Resources.Load(path));
}

but I need NNModel

cannot convert from 'Unity.Barracuda.Model' to 'Unity.Barracuda.NNModel'
FlorentGuinier commented 2 years ago

Hi @KamilMatejuk public static NNModel LoadModel1(string path) { Is expecting a serialize NNModel ressource aka a .bc asset. While you provide a .onnx file thus the problem. Two possible solution here:

Import the onnx file in editor and serialize the NNModel as a .bc asset that you load using the code above. Using NNModel.modelData.Value or alternatively ModelWriter.Save(filename,...)

Or Use the ONNXModelConverter.Convert(byte[] buffer) method to convert from onnx file data directly to a Model.

I'm closing the question as i feel it was answered :) Please reopen if needed!

Have a great day! Florent

jhlee0918 commented 2 years ago

@FlorentGuinier Hello~ I have same problem.

The model trained without problems. Assigning models by drag and drop in the Unity editor works without any issues. However, when I use the code below in my C# script, I get an error.

my code : byte[] modelValues = File.ReadAllBytes(_modelPath); Unity.Barracuda.NNModel nnmodel = ScriptableObject.CreateInstance(); nnmodel.modelData = ScriptableObject.CreateInstance(); nnmodel.modelData.Value = modelValues; nnmodel.name = "sa_agent";

                    rl_env.GetComponent<shooting_env>().AgentsList[0].Agent.GetComponent<BehaviorParameters>().Model = nnmodel;

Error Msg : NotSupportedException: Format version not supported: 244 Unity.Barracuda.ModelLoader+d11.MoveNext () (at Library/PackageCache/com.unity.barracuda@2.4.0-preview/Barracuda/Runtime/Core/ModelLoader.cs:223) Unity.Barracuda.ModelLoader.Load (System.IO.BinaryReader fileReader, System.Boolean verbose, System.Boolean applyPatching, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.4.0-preview/Barracuda/Runtime/Core/ModelLoader.cs:200) Unity.Barracuda.ModelLoader.Load (System.Byte[] stream, System.Boolean verbose, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.4.0-preview/Barracuda/Runtime/Core/ModelLoader.cs:140) Unity.Barracuda.ModelLoader.Load (Unity.Barracuda.NNModel model, System.Boolean verbose, System.Boolean skipWeights) (at Library/PackageCache/com.unity.barracuda@2.4.0-preview/Barracuda/Runtime/Core/ModelLoader.cs:31) Unity.MLAgents.Editor.BehaviorParametersEditor.DisplayFailedModelChecks () (at Library/PackageCache/com.unity.ml-agents@2.1.0-exp.1/Editor/BehaviorParametersEditor.cs:153) Unity.MLAgents.Editor.BehaviorParametersEditor.OnInspectorGUI () (at Library/PackageCache/com.unity.ml-agents@2.1.0-exp.1/Editor/BehaviorParametersEditor.cs:91) UnityEditor.UIElements.InspectorElement+<>cDisplayClass59_0.b__0 () (at <31768fe99cfe4466aa4a401169fb2ce5>:0) UnityEngine.GUIUtility:ProcessEvent(Int32, IntPtr, Boolean&)

FlorentGuinier commented 2 years ago

Seems the same looking at the callstack/code indeed. So probably same solution as above? Alternatively just use the simple way to load model: https://docs.unity3d.com/Packages/com.unity.barracuda@2.4/manual/Loading.html

jhlee0918 commented 2 years ago

@FlorentGuinier
Thanks for posting a reply. Your link is what I tried before.

In the last line of my code, the agent's BehaviorParameters.Model is Barracuda.NNModel.

Do you know how to convert Brracuda.model to Barracuda.NNModel?

                    var modelBin = Resources.Load<Unity.Barracuda.NNModel>(_modelPath);
                    var model = Unity.Barracuda.ModelLoader.Load(modelBin);

                    rl_env.GetComponent<shooting_env>().AgentsList[0].Agent.SetModel("basic_agent", model); // Error miss matching param type

Agent.SetModel( string behaviorName, Barracuda.NNModel model) // model is not nnmodel

FlorentGuinier commented 2 years ago

Do you know how to convert Brracuda.model to Barracuda.NNModel?

A NNModel is the byte stream representation of the model as serializable data. To be exact this byte[] can be accessed byNNModel.modelData.Value

A Model is the object oriented model as usable by Barracuda.

You can take an NNModel and "Load" it to get a Model using ModelLoader.Load(myNNModel);

I'm not sure converting back from the object oriented model to the serializable version is what you want, but if so you could use: ModelWriter.cs::Save() method. As we do when we import an ONNX file to serialize the NNModel asset see https://github.com/Unity-Technologies/barracuda-release/blob/b1eac6c34b12b1ef5506fb7121a29eda2997efd1/Barracuda/Editor/ONNXModelImporter.cs#L72.

jhlee0918 commented 2 years ago

@FlorentGuinier Thanks for replying again. The reason for insisting on NNModel is because it rejects the Model in the agent's behaviorParameters.(Image)

image

If this isn't work, I have to manually insert dozens or hundreds sessions. lol

                    byte[] model = File.ReadAllBytes(_modelPath);
                    var asset = ScriptableObject.CreateInstance<Unity.Barracuda.NNModel>();
                    asset.modelData = ScriptableObject.CreateInstance<Unity.Barracuda.NNModelData>();
                    asset.modelData.Value = model;

                    rl_env.GetComponent<shooting_env>().AgentsList[0].Agent.SetModel("basic_agent", asset);
FlorentGuinier commented 2 years ago

I think the best would be to ask ML-Agent what is the right approach there then. Maybe using there forum ?

jhlee0918 commented 2 years ago

@FlorentGuinier No, I don't use it. However, when I checked the call stack of the part where the error occurred, I confirmed that the error occurred in the question because the version of the model was low, and I am training the model again as a last resort.

I thought 244 was the error code number, but it wasn't, it was just the name of the Barracuda version.... I'm cursing myself for not being good at English.

To be honest, it's not a job I really like, and I think my boss might kill me.

FlorentGuinier commented 2 years ago

Unity.MLAgents.Editor.BehaviorParametersEditor.DisplayFailedModelChecks () (at Library/PackageCache/com.unity.ml-agents@2.1.0-exp.1/Editor/BehaviorParametersEditor.cs:153) The callstack above shows the ML-Agent package?

I confirmed that the error occurred in the question because the version of the model was low, and I am training the model again as a last resort.

I would advice to export an untrained (or very quickly trained) model while iterating on importing problems or on model architecture. Also without having the full context i feel the problem here is around the code here rather than the model. You could probably try any model actually to confirm maybe?

jhlee0918 commented 2 years ago

image

Thank you for your interest. Even after re-training, the same exception handling continues because the version of the model does not match here.

(The required version is 244, but the model I trained on is 20.)

jhlee0918 commented 2 years ago

@FlorentGuinier basic_agent.zip

This is the model onnx file where the problem occurred. (Not only this file, but all the files I just created without anything are not horde in my way. )

FlorentGuinier commented 2 years ago

@jhlee0918 As talked/discussed above i don't think the problem come from the model but from the code itself, so i don't think iterating on the model architecture is the right approach here.

jhlee0918 commented 2 years ago

@FlorentGuinier Thank you for answer. I'm try to find the problem in the code.

jhlee0918 commented 2 years ago

@FlorentGuinier I solved it! Thank you!!

uisng AssetDatabase model path : image

model load from assets path image

As you might expect, the code problem is correct. Thank you so much for your help.