Migrate to Ignition Gazebo

[srmainwaring]

New Feature

This issue is to track the migration of the wave simulation and hydrodynamics plugins to Ignition (Garden).

Desired behaviour

The wave visuals and physics behaviour are available in Ignition using the ogre2 render engine. There should be support for both Linux and macOS (the latter using Metal graphics API).

Implementation suggestion

The model and visual plugins from legacy Gazebo will be replaced with system plugins in Ignition. As before three plugins are required:

• [x] wave visuals system
• [x] wave physics system
• [x] hydrodynamics physics system

wave visuals

The wave visuals are perhaps the most challenging change as they require migration to the Ogre2 render engine and integration into the ignition-rendering framework. As with legacy Gazebo, there is no direct support for modifying meshes at run time, with the exception perhaps of the DynamicRenderable objects for Ogre and Ogre2 which sit outside the main mesh frameworks from ignition-common and ignition-rendering.

The proposed approach is to first create visuals for a static wave field, which can be done staying within the available mesh and rendering framework and will use the vertex generation from the existing OceanTile. Static waves will allow the following to be migrated:

• [x] wave generation methods

• [x] materials and shaders

• [x] parameter interfaces

  wave physics

• [x] Enable switching between the existing trochoid wave and development branch FFT wave algorithms

  hydrodynamics physics

• [x] No changes in functionality planned.

• [x] Clarify use of different vector and mesh types (CGAL vs Ogre vs Ignition)

Other changes

The migration to Ignition requires such a significant change, it could be regarded as a complete reworking of the library. Other changes that may be included as sub-issues are:

• [x] simplify the plugin / project names, reduce the number of projects
• [x] move from the catkin build system to colcon (i.e. strip out ROS dependencies)
• [x] factor out the core physics from the Ignition specific applications
• [x] review the use of CGAL for the mesh physics
• [x] look at performance improvements (specifically the ray intersection code which is the main bottleneck)
• [x] improved range of wave spectra
• [x] provide a standardised set of environment parameters with intermediate mappings to wave spectra and wave generator parameters

Progress

• https://github.com/srmainwaring/asv_wave_sim/pull/16
• https://github.com/srmainwaring/asv_wave_sim/pull/17
• https://github.com/srmainwaring/asv_wave_sim/pull/18
• https://github.com/srmainwaring/asv_wave_sim/pull/19
• https://github.com/srmainwaring/asv_wave_sim/pull/23
• https://github.com/srmainwaring/asv_wave_sim/pull/24
• https://github.com/srmainwaring/asv_wave_sim/pull/25
• https://github.com/srmainwaring/asv_wave_sim/pull/26
• https://github.com/srmainwaring/asv_wave_sim/pull/27

[srmainwaring]

Progress

This section will be periodically updated to track progress on this issue.

wave visuals

WIP on branch https://github.com/srmainwaring/asv_wave_sim/tree/feature/ign-garden-wip

Initial version of static FFT generated waves:

• Render engine: ogre2
• Graphics API: metal

Update 1

Investigated a number of options for dynamic meshes in Ignition with the Ogre2 render engine

• The existing approach of using Ogre::v1::MeshManager::createManual is not recommended Drawing a Flexible Object made of Vertices (FAST), further the Ogre2 render engine converts these to Ogre2 meshes using Ogre::Mesh::importV1 so the previous method for updating the mesh will not work (and will be slow).
• Ignition has a rendering::Ogre2DynamicRenderable object that does support triangle lists, however there are no index buffers used in this class, so that requires each tile vertex to be inserted multiple times (once for each triangle). Furthermore the class does not have methods to set the normals and tangent vectors (the normals are calculated internally, tangents are not calculated). An implementation using rendering::Ogre2DynamicRenderable is available as an example and to provide a base level benchmark. The clip below shows the technique, the waves look tessellated because each face has it's own vertices and they carry the normal for the face (i.e. there is no averaging of the normals)

https://user-images.githubusercontent.com/24916364/153723312-d753c838-ef08-49c2-a39c-765117b957b8.mov

• The most likely approach will be to write a new variant of rendering::Ogre2DynamicRenderable that adds texture coords and tangent vectors.

Update 2

Single OceanTile with dynamic FFT waves and averaged normals using a modified version of rendering::Ogre2DynamicRenderable as described above. Plenty of scope for increased efficiency in the implementation, but even an initial version is running acceptably well.

https://user-images.githubusercontent.com/24916364/153761799-449e933d-86fd-4b1f-b064-f38818195f85.mov

Update 3

wave shaders

Ignition Garden has support for custom shaders using the ShaderParam system plugin for the <visual> element. However it turns out that only one plugin is processed from the <visual> element (multiple may be specified, but only one is loaded).

There are a number of places where changes are required. The following may not be exhaustive:

• Only one plugin element is read in https://github.com/ignitionrobotics/ign-gazebo/blob/49d0673f318fd1d29dc1a75e9be5de2f829217ab/src/SdfEntityCreator.cc#L752-L761
• The message API does allows multiple plugins to be specified https://github.com/ignitionrobotics/ign-msgs/blob/a2cf3acfd5c1ef14c449ecffa2b31befe528061a/proto/ignition/msgs/visual.proto#L84
• The GzSceneManager only captures one plugin per visual https://github.com/ignitionrobotics/ign-gazebo/blob/49d0673f318fd1d29dc1a75e9be5de2f829217ab/src/gui/plugins/scene_manager/GzSceneManager.cc#L114-L148

A proper solution is to modify ignition-gazebo to support multiple visual plugin elements. In the interim a work-around is to re-implement the ShaderParams logic in the Wave system plugin.

Fix-it list

• Setting materials not behaving as expected
• Find out the correct way to add another visual to a link (or substitute the current one)
• As above, but replace the geometry instead
• How to get a new Entity from the ECM compared with say RenderUtils or SceneManager
• The normal calculation for primitives look wrong when using the ShaderParams system plugin - what is going on?
• Checks that normals, texture coords etc are being correctly calculated and passed, and where is this happening?

Update 4

A number of changes in order to support PBS materials. Custom shaders are still not working, but improved visuals are available using PBS materials which supply the bump map and environment map (the mixture of shallow and deep colours is not available with this approach).

https://user-images.githubusercontent.com/24916364/154754258-d1fa28dc-d86f-4d79-8a96-672b5d74c8c9.mov

• Added tangents to the dynamic mesh renderable (add a VES_TANGENT vertex element). The common::SubMesh does not store tangent vectors so we subclass and populate the extended common::SubMeshWithTangents instead. To use we pass the sub-mesh to common::Mesh using the std::move interface instead of the copy by value which ensures the stored sub-mesh retains it's vtable.
• Capture the material used by the wave model visual element (which is actually on the geometry in this case), and use it to set the material on the ocean visual.
• It does not appear necessary to create a a VES_BINORMAL vertex element, in fact if this is attempted Ogre will report a runtime error concerning attribute indices exceeding a permitted value (< 32).

Next Steps

• There is nice example of Ogre2 Ocean rendering using Terra here: Ogre 2.1 Ocean. Understand how to utilise the methods in the example with the wave generation methods used in the backend

[srmainwaring]

Initial working version

There are initial versions of all three plugins which allow objects of various shapes to interact with the wave field. There is much to be improved, but the bulk of the conceptual work required to migrate to Ignition is in place.

waves model plugin

The waves model system plugin holds a wavefield object and its parameters. The entity component system makes it simpler to manage these objects than in legacy Gazebo. A new component for the wavefield is used to store a weak pointer to the wavefield and then this is registered with a new entity. It can then be retrieved by name from the ECM.

This approach appears to work quite well, and eliminates the need for a separate wavefield entity object derived from the physics base model.

Outstanding tasks

• Read wave parameters from SDF
• Publish / subscribe to parameter messages from the transport layer
• Performance improvements (update is unreasonably slow - too many copies occuring when mapping between the different mesh system)

hydrodynamics plugin

The hydrodynamics system plugin is also substantially reworked from the original. It uses the ECM to retrieve the wavefield component registered by the wave model plugin. The collision shapes are obtained from the collision component using the same approach used in the buoyancy system plugin.

Outstanding tasks

• Read parameters from SDF
• Publish / subscribe to parameter messages from the transport layer
• Markers for water patch, waterline, submerged surface, etc are not implemented
• Missing components for pose and linear velocity of a link CoM - these have not yet been ported over from legacy Gazebo
• Missing checks that components are valid before accessing
• The wavefield name is hardcoded and should be set in parameters
• The weak_ptr should be locked for the entire update step
• The water patch is hardcoded because the collisions are missing an AABB interface

[srmainwaring]

Scaling to larger environments

https://user-images.githubusercontent.com/24916364/157044239-2d9ac4e2-746d-4824-b990-a5908a1e831b.mov

For larger models (ships) the ocean will need to be scaled accordingly, while retaining sufficient detail to make the local wave field interesting. Investigate the following:

• [x] Customise or develop a new Hlms specifically for an Ocean.
• [x] Move the vertex displacement for visuals entirely to the GPU using compute shaders
• [ ] Use compute shaders to calculate normals and tangents
• [x] Use textures to transport vertical and horizontal displacement maps.
• [x] Use the compositor to assemble the calculation
• [ ] Implement LOD while retaining period boundary conditions on the tile.

Some of the ideas listed above are investigated in this ogre-next branch: ogre-next - feature/v2-2-osrf-mac-ocean-compute2

Using textures to store the displacement, normal and tangent maps the trochoid wave example may be extended to a 10km x 10km region using 512m x 512m tiles. In this example the textures are updated in CPU and uploaded to GPU. The underlying vertex structure is a 128 x 128 segment Ogre plane repeated 2*10+1 times in each direction (i.e. 441 tiles).

https://user-images.githubusercontent.com/24916364/158489622-61f3901b-17a5-4e11-8c39-56b0ca7d45e4.mov

[srmainwaring]

FFT generated wave textures with 256m x 256m tiles and 200 x 200 grid segments using 256 x 256 FFT samples. There are some repetition artefacts which are mitigated to some extent by the tile size and viewpoint.

https://user-images.githubusercontent.com/24916364/169540114-8922989e-d9f0-445b-a75f-1907fdc1ac14.mov

[srmainwaring]

Closing as the main aspects of the initial migration have been dealt with in the following PRs: #16, #18, #19, #23, #24, #25, #26, #27, #28, #29, #31, #32, #33, #34, #36, #37, #38, #39, #40