search

laxnpander / OpenREALM

OpenREALM is a pipeline for real-time aerial mapping utilizing visual SLAM and 3D reconstruction frameworks.
GNU Lesser General Public License v2.1
462 stars 117 forks source link

Output results are missing/inconsistent #16

Closed sev-of-nine closed 4 years ago

sev-of-nine commented 4 years ago

Hi again,

Now that OpenREALM runs on my machine, I did some experiments using the demo data set provided.

Although I checked the *yaml-config-files in the realm_ros/profiles folder, the output of e.g. the ortho mosaic seems inconsistent. The stage_settings.yaml is the same for alexa_reco as well alexa_noreco but the latter writes the ortho image to disk, alexa_reco does not. There are other examples where this behaviour can be observed as well.

Can somebody verify this or is it just happening with my installation?

Any help is highly appreciated! Thank you!

laxnpander commented 4 years ago

@pjende: Hey Pjende, sorry I didn't see your post earlier. Could you resolve your issue? I am assuming that both approaches run according to your expectations and finish correctly, only the results are not saved? You can try to check the "save_ortho_rgb_all" or "save_ortho_gtiff_all" flag in the mosaicing stage settings. This should save incremental progress after every update.

PS: Remember this will take up lots of space because it will generate several 100 pictures every run.

sev-of-nine commented 4 years ago

Hi, I also have to apologise for coming back to you so late. I have tried changing the stage_settings.yaml for alexa_reco.launch but unfortunately without any success. The stage settings are the same for alexa_reco and alexa_noreco, the latter creates ortho tiles, though. Apart from the commented densification node, alexa_reco.launch and alexa_noreco.launch are identical.

Thank you for your help!

%YAML:1.0

type: mosaicing
queue_size: 5

th_elevation_min_nobs: 2
th_elevation_variance: 1.0

publish_mesh_every_nth_kf: 0
publish_mesh_at_finish: 0
downsample_publish_mesh: 0.5

Ortho
save_ortho_rgb_one: 0
save_ortho_rgb_all: 1
save_ortho_gtiff_one: 1
save_ortho_gtiff_all: 0

Elevation
save_elevation_one: 1
save_elevation_all: 0
save_elevation_var_one: 0
save_elevation_var_all: 0
save_elevation_obs_angle_one: 1
save_elevation_obs_angle_all: 0
save_elevation_mesh_one: 0

Etc
save_valid: 0
save_num_obs_one: 1
save_num_obs_all: 0

3D-Data
save_dense_ply: 0
laxnpander commented 4 years ago

@pjende: Hmmm, really weird. Can you run the alexa_reco and Post the log here? Either from the terminal or the log file in the mosaicing folder

sev-of-nine commented 4 years ago

Sure, the log from the terminal is extremely long and is mostly about pose estimation messages. This is the one from the mosaicing folder. Thank you!

File verbosity level: 9
date       time         ( uptime  ) [ thread name/id ]                   file:line     v| 
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]             loguru.hpp:2056     0| Logging to '/home/phil/aerial/openrealm_test2/catkin_ws/src/OpenREALM/realm_ros/output/20-02-19_09-28-17/mosaicing/stage.log', mode: 'a', verbosity: 9
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]         stage_base.cpp:49       0| Successfully initialized!
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]         stage_base.cpp:50       0| Stage path set to: /home/phil/aerial/openrealm_test2/catkin_ws/src/OpenREALM/realm_ros/output/20-02-19_09-28-17/mosaicing
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:394      0| ### Stage process settings ###
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:395      0| - publish_mesh_nth_iter: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:396      0| - publish_mesh_every_nth_kf: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:397      0| - do_publish_mesh_at_finish: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:398      0| - downsample_publish_mesh: 0.50
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:399      0| - use_surface_normals: 1
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:400      0| - th_elevation_min_nobs: 2
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:401      0| - th_elevation_var: 1.00
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:403      0| ### Stage save settings ###
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:404      0| - save_valid: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:405      0| - save_ortho_rgb_one: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:406      0| - save_ortho_rgb_all: 1
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:407      0| - save_ortho_gtiff_one: 1
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:408      0| - save_ortho_gtiff_all: 0
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:409      0| - save_elevation_one: 1
2020-02-19 09:28:18.421 (   0.327s) [        39CC9E80]          mosaicing.cpp:410      0| - save_elevation_all: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:411      0| - save_elevation_var_one: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:412      0| - save_elevation_var_all: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:413      0| - save_elevation_obs_angle_one: 1
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:414      0| - save_elevation_obs_angle_all: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:415      0| - save_elevation_mesh_one: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:416      0| - save_num_obs_one: 1
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:417      0| - save_num_obs_all: 0
2020-02-19 09:28:18.421 (   0.328s) [        39CC9E80]          mosaicing.cpp:418      0| - save_dense_ply: 0
laxnpander commented 4 years ago

@pjende: From this I can tell at least, that mosaicing is not receiving any frame messages at all. I assume that maybe the densification node crashes, can you post the log for this one too?

sev-of-nine commented 4 years ago

This is interesting, an output folder for 'densification' has never been created in any of the experiments I have conducted.

As it seems like, the node never runs...

[rviz-8] killing on exit
[realm_mosaicing-7] killing on exit
[realm_ortho_rectification-6] killing on exit
[realm_surface_generation-5] killing on exit
[realm_pose_estimation-3] killing on exit
[realm_exiv2_grabber-2] killing on exit
2020-02-19 12:19:05.419 ( 467.214s) [         365BE80] worker_thread_base.cpp:115      0| Thread 'Stage [pose_estimation]' received finish request...
2020-02-19 12:19:05.461 ( 467.257s) [        A11BD700] worker_thread_base.cpp:87       0| Thread 'Stage [pose_estimation]' finished!
2020-02-19 12:19:05.462 ( 467.257s) [         365BE80] worker_thread_base.cpp:115      0| Thread 'Publisher [pose_estimation]' received finish request...
2020-02-19 12:19:05.541 ( 467.336s) [        A19BE700] worker_thread_base.cpp:87       0| Thread 'Publisher [pose_estimation]' finished!
2020-02-19 12:19:05.601 ( 467.320s) [        AEA92E80] worker_thread_base.cpp:115      0| Thread 'Stage [surface_generation]' received finish request...
2020-02-19 12:19:05.604 ( 467.217s) [        A4F64E80] worker_thread_base.cpp:115      0| Thread 'Stage [ortho_rectification]' received finish request...
2020-02-19 12:19:05.607 ( 467.190s) [        7C87BE80] worker_thread_base.cpp:115      0| Thread 'Stage [mosaicing]' received finish request...
2020-02-19 12:19:05.607 ( 467.220s) [        5D2EB700] worker_thread_base.cpp:87       0| Thread 'Stage [ortho_rectification]' finished!
2020-02-19 12:19:05.685 ( 467.404s) [        62DF1700] worker_thread_base.cpp:87       0| Thread 'Stage [surface_generation]' finished!
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete

Please see again the launch file:

<?xml version="1.0"?>
<launch>
    <arg name="camera_id" default="alexa"/>
    <arg name="topic_adapter" default="/realm/$(arg camera_id)/input"/>
    <arg name="topic_pose_est" default="/realm/$(arg camera_id)/pose_estimation/frame"/>
    <arg name="topic_dense" default="/realm/$(arg camera_id)/densification/frame"/>
    <arg name="topic_surf" default="/realm/$(arg camera_id)/surface_generation/frame"/>
    <arg name="topic_rect" default="/realm/$(arg camera_id)/ortho_rectification/frame"/>
    <arg name="topic_mosaic" default="/realm/$(arg camera_id)/mosaicing/frame"/>

    <node pkg="realm_ros" type="realm_exiv2_grabber" name="realm_exiv2_grabber" output="screen">
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/input" type="string" value="/home/phil/aerial/openrealm_test/openrealm_data/open_realm_edm_dataset/edm_big_overlap_50p"/>
        <param name="config/rate" type="double" value="10.0"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
        <!--param name="config/opt/poses" type="string" value=""/-->
        <!--param name="config/opt/surface_pts" type="string" value=""/-->
        <!--param name="config/opt/set_all_keyframes" type="bool" value="True"/-->
    </node>

    <node pkg="realm_ros" type="realm_stage_node" name="realm_pose_estimation" output="screen">
        <param name="stage/type" type="string" value="pose_estimation"/>
        <param name="stage/master" type="bool" value="true"/>
        <param name="topics/input" type="string" value="$(arg topic_adapter)"/>
        <param name="topics/output" type="string" value="$(arg topic_pose_est)"/>
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
        <param name="config/method" type="string" value="orb_slam2"/>
    </node>

    <node pkg="realm_ros" type="realm_stage_node" name="realm_densification" output="screen">
        <param name="stage/type" type="string" value="densification"/>
        <param name="topics/input" type="string" value="$(arg topic_pose_est)"/>
        <param name="topics/output" type="string" value="$(arg topic_dense)"/>
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
        <param name="config/method" type="string" value="psl"/>
    </node>

    <node pkg="realm_ros" type="realm_stage_node" name="realm_surface_generation" output="screen">
        <param name="stage/type" type="string" value="surface_generation"/>
        <param name="topics/input" type="string" value="$(arg topic_dense)"/>
        <param name="topics/output" type="string" value="$(arg topic_surf)"/>
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
    </node>

    <node pkg="realm_ros" type="realm_stage_node" name="realm_ortho_rectification" output="screen">
        <param name="stage/type" type="string" value="ortho_rectification"/>
        <param name="topics/input" type="string" value="$(arg topic_surf)"/>
        <param name="topics/output" type="string" value="$(arg topic_rect)"/>
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
    </node>

    <node pkg="realm_ros" type="realm_stage_node" name="realm_mosaicing" output="screen">
        <param name="stage/type" type="string" value="mosaicing"/>
        <param name="topics/input" type="string" value="$(arg topic_rect)"/>
        <param name="config/id" type="string" value="$(arg camera_id)"/>
        <param name="config/profile" type="string" value="alexa_reco"/>
    </node>

    <node name="rviz" pkg="rviz" type="rviz" args="-d $(find realm_ros)/rviz/realm.rviz" />

</launch>
laxnpander commented 4 years ago

@pjende: It’s not about the launch file or settings, I am pretty sure about that. I can probably tell you what’s exactly the problem from the first ~30seconds of the terminal output. Some things are not printed to the log files unfortunately.

My guess: There is a problem with your CUDA setup. Did you run the catkin build according to the readme? So with the additional cuda flags? If yes and it builds fine, there still might be a cuda Problem of trying to use wrong versions and so on. Make sure everything is setup correctly for cuda!

laxnpander commented 4 years ago

Reopen if issue is not solved.