utterances-bot commented 1 year ago

Multibeam Sonar | Project DAVE

Physics-based modelling and simulation of Multibeam Echosounder perception in Gazebo Classic, Multibeam Sonar Simulation

https://field-robotics-lab.github.io/dave.doc/contents/dave_sensors/Multibeam-Forward-Looking-Sonar/

GriTRini commented 1 year ago

terminate called after throwing an instance of 'pcl::UnorganizedPointCloudException' what(): : Can't use 2D indexing with an unorganized point cloud Aborted [gazebo-2] process has died [pid 2122, exit code 134, cmd /home/jeong/catkin_ws/src/gazebo_ros_pkgs/gazebo_ros/scripts/gzserver -e ode /home/jeong/catkin_ws/src/sensor/worlds/empty.world name:=gazebo log:=/home/jeong/.ros/log/99600efe-e48e-11ed-bbd1-00155dfae7ae/gazebo-2.log]. log file: /home/jeong/.ros/log/99600efe-e48e-11ed-bbd1-00155dfae7ae/gazebo-2*.log

I ran blueview_p900_nps_multibeam_ray. But it doesn't run because the error works as shown. And when the blueview_p900_nps_multibeam model is executed, the sonar_image is displayed as a black screen and the same video as seen on YouTube is not displayed. What's the matter?

beifannaobaowei commented 8 months ago

Hello! I run roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch,but the windows doesn't appear.Could you help me solve it?

mabelzhang commented 8 months ago

@woensug-choi are you aware of this?

woensug-choi commented 8 months ago

any error msg on the console? did everything else worked? multibeam sonar requires specific installations

https://field-robotics-lab.github.io/dave.doc/contents/dave_sensors/Multibeam-Forward-Looking-Sonar/#installation

huangbintao188 commented 7 months ago

I also encountered the same problem as beifannaobaowei , the method I used Option B. Install on the Host

beifannaobaowei commented 6 months ago

I also encountered the same problem as beifannaobaowei , the method I used Option B. Install on the Host

you need to install the Dave first！

Nick-the-remaker commented 6 months ago

I got trouble mounting a multibeam sonar to a Rexrov, can someone tell me how to do?

mabelzhang commented 6 months ago

Could you elaborate on what you mean by "trouble"? What exactly did you encounter, is it at development time or runtime, etc.?

Nick-the-remaker commented 6 months ago

Thank you, I wang to use the multibeam sonar sensor on the Rexrov, I notices a file 'Rexrov_oberon7.xacro' contains these comments:

I tried them all, but it didn't work, there's no ROS topic about multibeam sonar, could you please tell me the right way to use a multibeam sonar sensor on a Rexrov?

Nick-the-remaker commented 6 months ago

those comments are:

   <xacro:uuv_forward_looking_sonar
      namespace="${namespace}"
      suffix="down"
      parent_link="${namespace}/base_link"
      topic="forward_sonar"
      mass="0.015"
      update_rate="40"
      samples="100"
        fov="1.54719755"
        width="768"
        height="492">
      <inertia ixx="0.00001" ixy="0.0" ixz="0.0" iyy="0.00001" iyz="0.0" izz="0.00001" />
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
      <visual>
        <geometry>
          <mesh filename="model://blueview_p900_nps_multibeam/meshes/p900.dae" />
        </geometry>
        <pose xyz="1.15 0 0.4" rpy="0 0.3 0"/>
      </visual>
   </xacro:uuv_forward_looking_sonar>

   <xacro:uuv_forward_multibeam_p900
      namespace="${namespace}"
      parent_link="${namespace}/base_link">
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
   </xacro:uuv_forward_multibeam_p900>

   <xacro:uuv_fl_multibeam_generic
      namespace="${namespace}"
      parent_link="${namespace}/base_link">
      <origin xyz="1.15 0 0.4" rpy="0 0.3 0"/>
   </xacro:uuv_fl_multibeam_generic>

   <!-- creates beam lines in Gazebo view --> 
   <xacro:uuv_forward_multibeam_sonar_m450_130 namespace="$(arg namespace)" parent_link="$(arg namespace)/base_link">
      <origin xyz="1.4 0 0.65" rpy="0 0 0"/>
   </xacro:uuv_forward_multibeam_sonar_m450_130>

mabelzhang commented 6 months ago

I edited the code block for readability.

I think this is a question for @woensug-choi

AmyPhung commented 4 months ago

I followed through the install instructions on this page with a fresh install of Ubuntu 22.04, CUDA 12.3, with a NVIDIA GeForce RTX 3070 card with driver version 535. Everything seems to run without any errors (nvidia-smi and nvcc --version also work as expected), but I just get a black image in the pop-up image window even when I put objects in front of the sonar in Gazebo. This happens for both the raster (roslaunch nps_uw_multibeam_sonar sonar_tank_oculus_m1200d_nps_multibeam.launch) and ray (roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam_ray.launch) versions. When I put objects in front of the sonar, the blue rays do appear to "hit" them, and the pointcloud shows the objects correctly when viewed in RVIZ. I used option B (git clone manually) for obtaining a copy of the repositories.

While running, I get printouts that look like this in the terminal:

Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1707942817.247347951, 102.471000000]: GPU Sonar Frame Calc Time 10/100 [s]

Any pointers for how to proceed? And do I need the full DAVE simulator built and running too, or is the multibeam simulator stand-alone? Looking through the code I didn't see anywhere the multibeam simulator was dependent on the DAVE repos, but I could be wrong

Nick-the-remaker commented 4 months ago

I followed through the install instructions on this page with a fresh install of Ubuntu 22.04, CUDA 12.3, with a NVIDIA GeForce RTX 3070 card with driver version 535. Everything seems to run without any errors (nvidia-smi and nvcc --version also work as expected), but I just get a black image in the pop-up image window even when I put objects in front of the sonar in Gazebo. This happens for both the raster (roslaunch nps_uw_multibeam_sonar sonar_tank_oculus_m1200d_nps_multibeam.launch) and ray (roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam_ray.launch) versions. When I put objects in front of the sonar, the blue rays do appear to "hit" them, and the pointcloud shows the objects correctly when viewed in RVIZ. I used option B (git clone manually) for obtaining a copy of the repositories.

While running, I get printouts that look like this in the terminal:
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1707942817.247347951, 102.471000000]: GPU Sonar Frame Calc Time 10/100 [s]
Any pointers for how to proceed? And do I need the full DAVE simulator built and running too, or is the multibeam simulator stand-alone? Looking through the code I didn't see anywhere the multibeam simulator was dependent on the DAVE repos, but I could be wrong

The multibeam sonar is stand alone, I edit some parts to combine the Rexrov and multibeam sonar, maybe it will be helpful to you.

in src/dave/urdf/dave_snippets/urdf/sensor_snippets.xacro, add:

<xacro:include filename="$(find nps_uw_multibeam_sonar)/urdf/multibeam_sonar_blueview_p900.xacro" />

in src/dave/urdf/robots/rexrov_description/urdf/rexrov_sensors.xacro, add:

<!-- Sonar sensor -->
<xacro:multibeam_blueview_p900_raster_macro name="blueview_p900" scale="'1 1 1'" sonar_image_topic="sonar_image" debug="false" sonar_image_raw_topic="sonar_image_raw" maxDistance="12" raySkips="10" plotScaler="0" sensorGain="0.4" writeLog="false" writeFrameInterval="5" artificialVehicleVibration="false" constantReflectivity="true" reflectivityDatabaseFile="variationalReflectivityDatabase" robot_link="${namespace}/base_link" ray_visual="true">
    <origin xyz="1.6 0 0.6" rpy="0 0 0"/>
</xacro:multibeam_blueview_p900_raster_macro>

in src/nps_uw_multibeam_sonar/urdf/multibeam_sonar_blueview_p900.xacro, edit this block:

<xacro:macro name="multibeam_blueview_p900_raster_macro"
               params="name scale sonar_image_topic debug
               sonar_image_raw_topic maxDistance raySkips plotScaler sensorGain writeLog
               writeFrameInterval artificialVehicleVibration constantReflectivity
               reflectivityDatabaseFile robot_link ray_visual">

add a '*origin' to this block:

<xacro:macro name="multibeam_blueview_p900_raster_macro"
               params="name scale sonar_image_topic debug
               sonar_image_raw_topic maxDistance raySkips plotScaler sensorGain writeLog
               writeFrameInterval artificialVehicleVibration constantReflectivity
               reflectivityDatabaseFile robot_link ray_visual *origin">

in src/dave/examples/dave_demo_launch/launch/dave_demo.launch, add:

<!-- image_view for sonar image -->
    <node name="image_view_sonar" pkg="image_view" type="image_view"
            args="image:=/rexrov/blueview_p900/sonar_image">
        <param name="window_name" value="blueview_p900" />
        <param name="autosize" value="true" />
        <param name="filename_format" value="/tmp/SonarImage_capture_%04i.jpg" />
    </node>

    <node pkg="tf" type="static_transform_publisher" name="multibeam_sonar_base_link" args="0 0 0 0 0 0 world blueview_p900_link 50" />

Finally, run roslaunch dave_demo_launch dave_demo.launch
good luck

AmyPhung commented 4 months ago

Thanks for the pointers. Unfortunately, I'm still getting the same issue of the sonar image showing up blank, even after installing Dave. Last time I went through the install instructions I used option B (git cloning the repos), and when I ran roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch a tank with objects didn't show up like they did in the instructions. I wiped everything and re-did the install, using option a (vcs import) instead. This time, when I ran the same command the tank showed up but the sonar is still black. I don't get any errors in the terminal, but I'm still not getting the sonar image to show anything useful. Here's how I'm running the sim:

. ~/rocker_venv_cuda/bin/activate
cd ~/rocker_venv_cuda/dockwater
./run.bash -c dockwater:noetic
source ~/uuv_ws/devel/setup.bash
roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch

Here's the terminal output:

roslaunch nps_uw_multibeam_sonar sonar_tank_blueview_p900_nps_multibeam.launch
... logging to /home/chaoslab/.ros/log/72a3d392-d1c1-11ee-94af-0242ac110002/roslaunch-265f69df5edc-1957.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://265f69df5edc:34517/

SUMMARY
========

PARAMETERS
 * /gazebo/enable_ros_network: True
 * /image_view_sonar/autosize: True
 * /image_view_sonar/filename_format: /tmp/SonarImage_c...
 * /image_view_sonar/window_name: blueview_p900
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /use_sim_time: True

NODES
  /
    gazebo (gazebo_ros/gzserver)
    gazebo_gui (gazebo_ros/gzclient)
    image_view_sonar (image_view/image_view)
    multibeam_sonar_base_link (tf/static_transform_publisher)

auto-starting new master
process[master]: started with pid [1965]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 72a3d392-d1c1-11ee-94af-0242ac110002
process[rosout-1]: started with pid [1975]
started core service [/rosout]
process[gazebo-2]: started with pid [1982]
process[gazebo_gui-3]: started with pid [1987]
process[image_view_sonar-4]: started with pid [1992]
process[multibeam_sonar_base_link-5]: started with pid [1993]

(blueview_p900:1992): dbind-WARNING **: 15:32:08.463: Couldn't connect to accessibility bus: Failed to connect to socket /run/user/1000/at-spi/bus_1: No such file or directory
[ INFO] [1708633928.835523646]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1708633928.836481996]: waitForService: Service [/gazebo/set_physics_properties] has not been advertised, waiting...
[ INFO] [1708633928.904529434]: Finished loading Gazebo ROS API Plugin.
[ INFO] [1708633928.905264577]: waitForService: Service [/gazebo_gui/set_physics_properties] has not been advertised, waiting...
amdgpu: os_same_file_description couldn't determine if two DRM fds reference the same file description.
If they do, bad things may happen!
[ INFO] [1708633929.265721333]: waitForService: Service [/gazebo/set_physics_properties] is now available.
[ INFO] [1708633929.275588218]: Physics dynamic reconfigure ready.
[ INFO] [1708633931.234422903]: 
[ INFO] [1708633931.234468017]: ==================================================
[ INFO] [1708633931.234477234]: ============   SONAR PLUGIN LOADED   =============
[ INFO] [1708633931.234484849]: ==================================================
[ INFO] [1708633931.234491982]: ============      RASTER VERSION     =============
[ INFO] [1708633931.234498855]: ==================================================
[ INFO] [1708633931.234515326]: Maximum view range  [m] = 10
[ INFO] [1708633931.234526767]: Distance resolution [m] = 0.0501672
[ INFO] [1708633931.234534541]: # of Beams = 512
[ INFO] [1708633931.234542847]: # of Rays / Beam (Elevation, Azimuth) = (228, 1)
[ INFO] [1708633931.234550722]: Calculation skips (Elevation) = 10
[ INFO] [1708633931.234557554]: # of Time data / Beam = 399
[ INFO] [1708633931.234571721]: Reflectivity method : Constant
[ INFO] [1708633931.234582872]: ==================================================
[ INFO] [1708633931.234594774]: 
[ INFO] [1708633931.234614471]: Raw data at /tmp/SonarRawData_{numbers}.csv
[ INFO] [1708633931.234631833]: every 5 frames
[ INFO] [1708633931.234647122]: 
[ INFO] [1708633931.239816571]: Camera Plugin: The 'robotNamespace' param did not exit
[ INFO] [1708633931.242668940]: Camera Plugin (ns = )  <tf_prefix_>, set to ""
GPU Sonar Computation Time 146/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.090993722, 1.842000000]: GPU Sonar Frame Calc Time 153/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.382715677, 2.134000000]: GPU Sonar Frame Calc Time 9/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.561362956, 2.312000000]: GPU Sonar Frame Calc Time 10/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.745047807, 2.496000000]: GPU Sonar Frame Calc Time 10/100 [s]

GPU Sonar Computation Time 2/100 [s]
Sonar Ray Summation 5/100 [s]
GPU Window & Correction 1/100 [s]
GPU FFT Calc Time 0/100 [s]
[ INFO] [1708633933.926459492, 2.677000000]: GPU Sonar Frame Calc Time 10/100 [s]

And here's what shows up: blueview-blank

I followed @Nick-the-remaker 's instructions to add the multibeam to Dave, which also runs without errors but results in the same issue blueview-blank-dave

I suspected there might be something wrong with CUDA or my NVIDIA drivers, but as far as I can tell those seem to be working properly. From inside the container: nvcc --version produces:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Wed_Nov_22_10:17:15_PST_2023
Cuda compilation tools, release 12.3, V12.3.107
Build cuda_12.3.r12.3/compiler.33567101_0

and from the container, nvidia-smi produces:

nvidia-smi
Thu Feb 22 15:38:27 2024       
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.154.05             Driver Version: 535.154.05   CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 3070 ...    Off | 00000000:01:00.0 Off |                  N/A |
| N/A   39C    P0              27W /  80W |      8MiB /  8192MiB |      0%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
+---------------------------------------------------------------------------------------+

Any additional pointers for getting the multibeam sim to run would be appreciated!

LiliHU-tech commented 2 months ago

When I use the GPU version of the sonar to change the maximum detection distance ' maxDistance' to 300, the program will crash inexplicably, how to solve it. And when I change the maximum detection distance to 100, the length of the sonar image becomes longer, but the width cannot fully display the image with an opening angle of 90 degrees, only a small part of it can be displayed

Field-Robotics-Lab / dave

Multibeam Sonar | Project DAVE #267

Multibeam Sonar | Project DAVE