srmainwaring / asv_wave_sim

This package contains plugins that support the simulation of waves and surface vessels in Gazebo.
GNU General Public License v3.0
118 stars 32 forks source link

FFT Waves #10

Closed JuliusHendrix closed 2 years ago

JuliusHendrix commented 3 years ago

Hey Rhys,

First of all I wanted to thank you again for creating this package; it has been one of the foundations of a project I've been doing, and it has not disappointed, so thanks a lot!

I recently discovered your fft_waves branch and have been playing with the concept of implementing a complex wavefield with Fourier Transforms. I would like to try to implement it in my project, but I have two questions:

First of all, I see that since the branch is still experimental, there is no possibility to change the wavefield. To be able to change this wavefield to a combination of different waves is an obvious benefit, so I was wandering what files I would have to alter to try to implement a way to change the wave frequencies. I found the following file, but there may be more connections which I'm missing:

WaveSimulationFFTW.cc

Second, I was wondering about the performance of the fft_waves code. The code from the master branch has proven useful, but the performance I've been getting has not been super. I run the simulation in a Virtual Machine, so this probably reduces performance somewhat. To give an example: In an ocean world with no waves, I place 3 meshes with buoyancy. Two with ~300 faces and one with ~800 faces. I get a realtime factor of ~0.2, and when I publish a wave message, it drops quite a bit further. How does the performance of the fft_waves branch compare to that of the main?

Thanks,

Julius

srmainwaring commented 3 years ago

@JuliusHendrix I'm glad it's proving useful. I've not been active on the repo code recently but have some WIP regarding the FFT generated wave fields. The main gap is implementing a nice solution for travelling waves with good control over the wave direction and distribution functions. The plan is to implement some of the methods discussed here: https://wikiwaves.org/Ocean-Wave_Spectra and here: https://hal.archives-ouvertes.fr/file/index/docid/307938/filename/frechot_realistic_simulation_of_ocean_surface_using_wave_spectra.pdf

I have Python prototypes for the changes in Jupyter notebooks but have not yet transferred them into the plugin. This approach should provide a more natural parameterisation for setting wave conditions.

The FFT waves are configured in two places - one for the physics:

https://github.com/srmainwaring/asv_wave_sim/blob/0d9f6d44e60e35351154f9aa58d2926b4d1169c6/asv_wave_sim_gazebo_plugins/src/Wavefield.cc#L327-L330

and the other for the visual:

https://github.com/srmainwaring/asv_wave_sim/blob/0d9f6d44e60e35351154f9aa58d2926b4d1169c6/asv_wave_sim_gazebo_plugins/src/OceanVisual.cc#L133-L136

The way to keep them synchronised is to have callback functions update the parameters when a custom message describing the wave parameters is received. I did not want to introduce another message type (which is a public interface) until I had the interface for specifying the wave distribution finalised - hence the hardcoding.

There are a couple of execution paths for the FFT, one uses FFTW and the other and OpenCL implementation. The feature/ardupilot_sailboat_poc branch which I've built a docker container for runs well on an 4 core 8MB RAM Ubuntu VM using the FFTW simulation (the extra RAM is the critical part in getting the performance). The configuration is in the OceanTile ctor:

https://github.com/srmainwaring/asv_wave_sim/blob/70f2916d018585cac4fd9b94171821d117b87dbd/asv_wave_sim_gazebo_plugins/src/OceanTile.cc#L40-L90

There is additional overhead with the FFT waves compared to the trochoid waves, it's mitigated a bit because the new branch supports tiling which reduces the need to have a super large mesh for the ocean. However the wave generation is not the performance bottleneck - that turns out to be the ray-casting needed to project each vertex from the buoyant object into an ocean mesh triangle (I have the instrumenting results somewhere and it is by far an above the most significant cost in the simulation). This part of the code is not parallelised and I think adding TBB or something similar there would be the quick win (a GPU ray caster would be even better - but small steps!).

My current thinking is to factor the wave code and hydrodynamics code into separate (non-Gazebo specific) libraries that can operate standalone, perhaps with python bindings using pybind11. I think that would aid in performance testing and experimenting with different wave generation functions as I could use the models in Python and Juypter etc.

In the meanwhile here is a clip from a project where I've been using the FFT wave version to develop a custom ArduPilot mixer and Gazebo model for a jet-boat with reversible thrusters. The boat is moving into a loiter circle while subject to a 5m/s headwind:

https://user-images.githubusercontent.com/24916364/110970046-42b9e200-8351-11eb-958a-bac243d3665d.mov

JuliusHendrix commented 3 years ago

Hey @srmainwaring,

Thanks for your extensive reply. I've taken a look at the articles you linked and the generated wavefields look impressive.

I'm not too familiar with your code (or C++ in general...), but am I correct in saying that this is where the magic happens?

https://github.com/srmainwaring/asv_wave_sim/blob/70f2916d018585cac4fd9b94171821d117b87dbd/asv_wave_sim_gazebo_plugins/src/WaveSpectrum.cc

Where the wind velocity gets converted to a wave spectrum? There are a few steps in the entire process which I don't follow, this seems to be the method described in the articles you sent before.

It seems you have already done a lot of work on this, which is fantastic!

Your performance also seems so be much better than mine. Does this have to do with the tiling you mentioned? Right now I'm running a virtual machine with ~28GB ram and 16 cores, which should be plenty by the sound of it. If you think switching to this branch will also help performance-wise, I have even more reason to do so.

Anyways, this project looks great and I can't wait to see what you do next.

Cheers,

Julius

srmainwaring commented 3 years ago

Hi @JuliusHendrix, I'd expect the simulation to fly on a machine of that spec - certainly achieve near RT performance on the meshes of size you describe on either master or the develop branch. Without knowing more about how your set up is configured it is hard to pin down exactly where the bottleneck is, but if you are able to share the mesh files / models I can take a look and try to replicate the sim to see where things are slowing down. One possibility is the mesh contains some degenerate triangles - these can cause a problem as if they are generating NaNs in the code they will propagate until the point at which the force is applied to the rigid body at which point they are filtered out. This can drastically affect the simulation performance as propagating NaNs is very slow.

srmainwaring commented 2 years ago

Hi @JuliusHendrix - I'll be tracking the improvements to the FFT generated waves in #22. The changes will be for the plugin targeting Ignition Gazebo rather than Gazebo11 (I'm planning to put support for the latter in maintenance / archive at some point).