Ogre background loader thread blocks main thread.

[juj]

When running async thread loading, the main thread blocks on several occasions to wait for the asset loader thread. This causes performance issues at 100ms-1000ms scale while assets are being loaded.

The blocking occurs on this mutex https://bitbucket.org/sinbad/ogre/src/2ba0595e05fb/RenderSystems/Direct3D9/include/OgreD3D9HardwarePixelBuffer.h#cl-78 . Interestingly, the OpenGL Render System implementation on Ogre does not contain this kind of global system-wide mutex.

Possible workarounds:

• The EC_Mesh raycast code causes Ogre to acquire the mutex in the main thread. Specify the selection layer as 0 for all placeables which do not need to be clicked on, so that the raycast code is not executed that often (mostly for static unclickable geometry)
• The Hydrax component causes the main thread to acquire the mutex each frame. Remove Hydrax components from the scene, and use a WaterPlane component instead.
• Creating resources (like materials, or other overlays) in the main thread will probably cause the Ogre to acquire the mutex. Don't do this while assets are being loaded asynchronously.
• Switch to OpenGL. This might not help much, unless Ogre implements the use of wglShareLists (see http://www.opengl.org/wiki/OpenGL_and_multithreading )

Possible fixes:

• Implement manual asynchronous asset loading code which loads in assets without resorting to code that acquires a system-wide mutex.
• Implement a shadow local copy of the geometry data in Mesh assets to avoid having to use Ogre system-wide mutexes. Ogre allows a shadow copy for hardware buffers, but this is not available if Ogre mesh manager load functions are used.
  • Additionally rewrite all other parts of the main thread which utilize Ogre calls that acquire the Ogre system mutex. E.g. for Hydrax which streams in dynamic GPU data per-frame, this can be difficult.
  • Fix Ogre to not to resort to system-wide mutexes.

[jonnenauha]

Have you been thinking how to make the custom ogre threaded loading? I read the ogre forums about this and many people seem to have the exact same mutex lock problem we are having, even with queued loading it blocks rendering and biggest issue for people is textures (that are understandably heavy to decode etc.).

How can we do threaded loading and access Ogre from it without its mutexes? Or is this a matter of doing some prep work, like getting .mesh file data to buffers or unpacking encoded textures to raw pixel data, before accessing/passing this data to Ogre? Are there some DirectX APIs that we could leverage to upload stuff to the gpu asynch and then telling Ogre about it after the fact?

Reg. opengl: The ogre forum folks seemed to report that opengl indeed blocked less. Personally in our Admino builds when opengl is enabled I don't see much difference to the dx rendering, and if I can't see the diff visually its probably not much.

[juj]

OpenGL indeed does not have the OGRE_STATIC_MUTEX_INSTANCE(D3D9HardwarePixelBuffer::msDeviceAccessMutex) and OGRE_STATIC_MUTEX_INSTANCE(D3D9Resource::msDeviceAccessMutex) objects. Although the necessity of those mutexes is questionable.