Object reference not set to instance at Unity Barracuda, Reality Collective Service

[LucasJSR]

Hello, thank you for this amazing project! I'm trying to add a custom YOLO model to the project in order to run it on HoloLens 2. However, when opening the project on Unity, I'm getting the following error:

"Object reference not set to an instance of an object at Unity.Barracuda.ModelLoader.Load (Unity.Barracuda.NNModel model, System.Boolean verbose, System.Boolean skipWeights) [0x00000] in C:\Users\jose\OneDrive\Software\YoloHolo\YoloHolo\Library\PackageCache\com.unity.barracuda@557e933876\Barracuda\Runtime\Core\ModelLoader.cs:31 at YoloHolo.Services.YoloProcessor.Initialize () [0x00000] in C:\Users\jose\OneDrive\Software\YoloHolo\YoloHolo\Assets\Application\Scripts\Services\YoloProcessor.cs:25 at RealityCollective.ServiceFramework.Services.ServiceManager.InitializeAllServices () [0x00033] in C:\Users\jose_\OneDrive\ Software\YoloHolo\YoloHolo\Library\PackageCache\com.realitycollective.service-framework@1.0.1\Runtime\Services\ServiceManager.cs:1327 UnityEngine.Debug:LogError (object) RealityCollective.ServiceFramework.Services.ServiceManager:InitializeAllServices () (at Library/PackageCache/com.realitycollective.service-framework@1.0.1/Runtime/Services/ServiceManager.cs:1332) UnityEditor.EditorApplication:Internal_CallDelayFunctions ()"

I haven't been able to fix the issue, and because of this I'm unable to build the project. Do you know what I could be missing here?

Thanks in advance.

[LocalJoost]

I have no idea what branch you pulled, but have you read this or this? The v7 model is not included, because it was published by someone else.

[LucasJSR]

I’ve pulled the main branch. Yes, I’ve read your blog post and I’ve got the yolov7-tiny_256x320.onnx file from the tar file you’ve linked to. I tried using this onnx file as well as the custom onnx file I’ve built, but both of them return the error I’ve mentioned before.

[LocalJoost]

Okay, I will try a fresh pull myself later, but I will give no one tip: never put a Unity project on a OneDrive folder. I tried it years ago and it played havoc with your project because of the messy way Unity handles files. Also, don't use such a long folder name, that might be an issue. Or make sure you have enabled long file names . Try to put in in C:.\projects

[LocalJoost]

So I just did a fresh pull into c:\projects, so the project sits in c:\projects\YoloHolo, and did not have any problem opening the project. I assume you open it with the same version as I created it?

[LucasJSR]

Putting the project in the C:/ drive worked well for the YOLOv7 model you've used, thank you. However, I'm still getting errors when I try to use my custom model. I'm getting the same "Object reference not set to an instance of an object" message I mentioned above, as well as two error messages related to the importing of my custom model.

One of them says "OnnxImportException: Unexpected error while parsing layer /model.105/Split_output_0 of type Split. Unsupported default attribute split for node /model.105/Split_output_0 of type Split. Value is required.", while the other says "Asset import failed, "Assets/Application/Model/yolov7-class2.onnx" > OnnxImportException: Unexpected error while parsing layer /model.105/Split_output_0 of type Split. Unsupported default attribute split for node /model.105/Split_output_0 of type Split. Value is required."

Do you know what could be the cause? I've followed this tutorial for training the custom model: https://blog.roboflow.com/yolov7-custom-dataset-training-tutorial/

Thanks again for the help.

[LocalJoost]

I am sorry, my expertise is Unity and Mixed Reality Development, I don't have much in depth knowledge of ONNX and YOLO model innards. Obviously your model is not what Unity Barracuda expects, but what exactly the issue is, no idea.

[LocalJoost]

Maybe this is a clue? https://docs.unity3d.com/Packages/com.unity.barracuda@1.0/manual/Exporting.html

[LucasJSR]

Sorry for the late reply, and thank you for the help! I was able to make my custom model work with the software, but I'm having some trouble with the alignment of the labels with the real object. The labels seem to be rendered a bit far from the object. As the custom model was developed to detect certain cable connectors, I wonder if it has to do with the fact that the connector is a smaller object, while the original software and AI model was made to detect larger object such as TVs and books? Do you think this could be caused due to the size of the bounding boxes?

[LocalJoost]

@LucasJSR Great to hear you got it working. As to the 'far from the object'- I think it may have to do with the smaller sizes, but also the viewing angle. For locating, the software takes a stab (almost literally) at the spatial map. If the objects are too small to be detected by the depth cameras and there is nothing substantial close 'behind' it, it might not be able to properly locate it. If you remember my airplane POC: often label was a bit next to it, because although on the image the X/Y location was detected correctly, for the Z location it actually hit the floor where the aircraft model is sitting on.

Can you send me a picture or a short movie? If that is too privacy, sensitive, just ping me direct only one of the contact possibilities on my blog

[LucasJSR]

@LocalJoost Sorry for the (very) late reply once again. I've been talking with my team about the labels, and we decided to use a different approach - a fixed position custom label on the top of the scene - as it would be more adequate for our project.That said, I still would like to ask you for help on a different topic, if you don't mind.

As the original issue I've opened here has already been solved, I think it would be better for you to close this issue, and I could contact you through your e-mail, or maybe your Discord. Would that be OK?

Once again, thank you for your help!

[LocalJoost]

That would be very OK. You can close the issue as well if you like 😁