mcpiroman / UnityNativeTool

Allows to unload native plugins in Unity3d editor
MIT License
183 stars 18 forks source link

Crash with IUnityInterface ? #42

Closed AnimatorJeroen closed 6 months ago

AnimatorJeroen commented 6 months ago

Thank you for this amazing package.

Unfortunately I can't get it to work with this unity native plugin sample project: https://github.com/Unity-Technologies/NativeRenderingPlugin

Everything compiles but the Unity editor freezes up when loading the dll. When I start playmode or when I press alt - shift - D.

(WIthout the package the dll loads correctly and works in playmode). I followed all the steps under the "usage" section.

Im running Unity 2022.3.5f1 on Windows x64.

Could the problem be that the dll uses IUnityInterface.h? for instance: a function is declared like this: extern "C" void UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API UnityPluginLoad(IUnityInterfaces* unityInterfaces)

I'm not sure if this is supported?

I also tried specifying the mocked functions, this results in: image

However on alt + shift + D it freezes up.

Do you have any idea what could be causing the issue?

Thanks in advance Jeroen

mcpiroman commented 6 months ago

Hi! Yes, native plugins should be supported, see https://github.com/mcpiroman/UnityNativeTool?tab=readme-ov-file#low-level-interface-callbacks-support

But I haven't played much with this scenario, and it may happen that Unity did some breaking change since. So if you still have this problem, please attach a .NET debugger with native code debugging enabled (or just a native debugger) and see what the hanging frame is.

AnimatorJeroen commented 6 months ago

Thank you for the quick feedback! I added the debugger from VS and noticed that it's an error somewhere in openGL Renderer API. Using D3D11, the DLL manipulator script works!

I will investigate the opengl issue further and post the solution here if I find it. Note: I had to append: __ to the build path in VS to make VS attach the debugger to the right dll. image

AnimatorJeroen commented 6 months ago

And thanks again for this awesome solution. This will speed up my dev time by a factor 10 :)

AnimatorJeroen commented 6 months ago

So there seems to be some initialization order issue that only happens with openGL render api, and dynamic dll loader.

In OnPluginLoad(), The function OnGraphicsDeviceEvent(kUnityGfxDeviceEventInitialize); is called once in order to not miss the event.

This then initializes gl3wInit() with the current openGL context.

However when loading the plugin using dynamic dll loader, at this stage the GL_Context is 0.

I tried to postpone the gl3wInit() bit to later, but to no avail. Switching to glew did not solve the problem. Maybe the GL_Context is not on the same thread when called from the dynamic dll loader?

mcpiroman commented 6 months ago

Yeah, I think I only tested that those native callbacks do fire, in general resemblance to how they do under vanilla unity. But I don't deny that there may be some quirks, such as a missed order. There might be issues with threading too. I do not spawn or switch between threads, but it may be that Unity handles scripts and native plugins in different threads internally. I remember Unity being rather singlethreaded, but it may have changed since.

I think my mechanism for handling those native plugins was really simple, there is just one or two special native functions being called, so the problem may likely be somewhere in-between. But I don't use Unity anymore, so I must leave the investigation to you unfortunately. If you debugged though then you may know my code better then I do now anyway.

AnimatorJeroen commented 6 months ago

Ha, finally solved it. So it was indeed a threading issue:

From: https://www.codeproject.com/Articles/1216876/Unity-Graphics-Emulator-for-Native-Plugin-Developm "NB: The most important thing about OpenGL mode is that Unity uses multiple GL contexts initialized by different threads. As a result, OnRenderEvent() may not be called from the same thread as OnGraphicsDeviceEvent(), which means that GL context-specific objects such as VAO, FBO, and program pipelines cannot be initialized in OnGraphicsDeviceEvent()."

I ended up simply moving the openGL Initialization code to the OnRenderEvent so it has the GL_Context. Then setting a static bool s_GraphicsInitialized to true so it only happens once.

mcpiroman commented 6 months ago

Glad that you solved it quickly. I don't really recall much about those OnRenderEvent etc. functions, and what did I do to make them work, if anything at all, only that I probably deemed them supported. It seems that this project only explicitly calls UnityPluginLoad and UnityPluginUnoad, and it is done from the scripting thread. Maybe those rendering functions are then called from the scripting thread too, which is not what Unity expects. So I don't really know how I can correct anything from my side, though suggestions are welcome.