srmainwaring
commented
3 years ago This branch (https://github.com/srmainwaring/ign-gazebo/tree/feature/ign-gazebo5-macos) contains proof of concept changes to support the sensors system plugin and GUI rendering on macOS.
There is some platform / render engine specific code dealing with OpenGL contexts that needs to be moved elsewhere but the changes illustrate the proposed approach and does render a camera view in the image display (but still seems quite fragile as seemingly minor change to a SDF file will cause a crash).
iche033
johntron
scpeters
Desired behavior
Would like the sensors system plugin to run on macOS using the ogre2 render engine.
In the following we'll use the
camera_sensor.sdfworld as an example, but the discussion applies to any model loading the sensor system plugin with the SDF element:If you run the following command on Ignition Ediface you'll most likely see a crash on macOS:
By including additional exception handling in Ogre2 we see that the reason is that the
Sensorsclass is trying to create a new render window on a render thread. The requested operation is not allowed in the Cocoa window system, where there are some restrictions that require the initialisation ofNSWindowand manipulationNSViewto occur on the main thread. A full stack trace is included below in the Additional context section.The new feature then is the ability to have some of the initialisation of the sensor system occur on the main thread before control is ceded to the render thread.
Alternatives considered
Modify the Ogre2
CocoaWindowclass to support creating windows on secondary threads. The Apple developer guide has this document on Thread Safety. It suggests that aNSWindowcan be created on a secondary thread (but is not generally thread-safe). However all operations on aNSViewmust occur on the main thread. Ogre makes use of both objects.The function call raising the exception in Ogre is:
OgreGL3PlusWindowis aNSWindow, so the initialisation of aNSWindowis failing in this case contrary to the comments in the Apple developer guide, and it's not clear that there is an alternative way to initialise a new window for the case at hand.Implementation suggestion
The challenge is that the initialisation of the render system in
Sensorsis deferred and occurs in:https://github.com/ignitionrobotics/ign-gazebo/blob/9fbb9b988c20711fcff706fb14d034516e027e6b/src/systems/sensors/Sensors.cc#L180-L206
which is already running on a render thread.
It is ultimately triggered by
this->dataPtr->doInitbeing set totruein:https://github.com/ignitionrobotics/ign-gazebo/blob/9fbb9b988c20711fcff706fb14d034516e027e6b/src/systems/sensors/Sensors.cc#L427-L451
which in turn is called from a worker thread in the
SimulationRunner.A suggestion that would keep deferred initialisation and avoid blocking in the main thread is to provide another system interface, say
ISystemInitthat theSimulationRunnerwould guarantee to call on the main thread after the system was configured.I'm happy to attempt to implement and test this proposal but wanted to sound out the development team on the design in case there are major issues with what is being suggested.
Update
Since
SimulationRunner::UpdateSystemscalls theISystemPreUpdateandISystemUpdateinterfaces from the main thread we could initialise the render engine fromSensors::Updateor subclassSensorsfromISystemPreUpdatethen useSensors::PreUpdaterather than introduce a new interface.Additional context
This is the
camera_sensors.sdfexample running on macOS with the sensor system plugin disabled:This is the log output from the server process with the plugin enabled. The code is from the
ign-gazebo5branch modified to support macOS with additional error reporting enabled inign-rendering5andogre-next2.1: