search

uzh-rpg / rpg_dvs_evo_open

Implementation of EVO (RA-L 17)
164 stars 43 forks source link

Error during ''live processing'' with DAVIS 346 #6

Open C4sp1 opened 3 years ago

C4sp1 commented 3 years ago

Hi! I am currently trying to run the pipeline in live mode on ROS-Melodic, Ubuntu 18.04, using DAVIS 346 sensor. I tried to run it with the ''flyingroom rosbag'' and the pipeline started without any errors. When I run it in live mode, SVO node always crushes right away.

I did the following steps:

where DAVIS-00000336 is the same calibration file that is saved in the .ros/camera_info folder.

Every time I run roslaunch I get the following error:

[svo-1] process has died [pid 9955, exit code -11, cmd /home/c4sp1/catkin_ws/devel/lib/svo_ros/svo_node events:=/dvs/events remote_key:=/evo/remote_key __name:=svo __log:=/home/c4sp1/.ros/log/a996bfae-f69f-11eb-b50f-b07b25408b14/svo-1.log].
log file: /home/c4sp1/.ros/log/a996bfae-f69f-11eb-b50f-b07b25408b14/svo-1*.log

I also get this warning nevertheless the distorsion model is specified in the calibration file.

[ WARN] [1628256826.590702738]: Camera calibration file did not specify distortion model, assuming plumb bob

Any ideas on what might be the issue?

Thanks

I attach all the terminal output which generate the error

c4sp1@c4sp1-G5-5500:~/catkin_ws/src$ roslaunch dvs_tracking live.launch auto_trigger:=false camera_name:=DAVIS-00000336 events_topic:=/dvs/events
... logging to /home/c4sp1/.ros/log/a996bfae-f69f-11eb-b50f-b07b25408b14/roslaunch-c4sp1-G5-5500-9928.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://c4sp1-G5-5500:45783/

SUMMARY
========

CLEAR PARAMETERS
 * /svo/

PARAMETERS
 * /calib_file: /home/c4sp1/catki...
 * /camera_name: DAVIS-00000336
 * /dvs_bootstrap_frame_id: /camera_0
 * /dvs_bootstrapping/activation_threshold_min: 10
 * /dvs_bootstrapping/activation_threshold_patch_size: 13
 * /dvs_bootstrapping/adaptive_thresholding: True
 * /dvs_bootstrapping/auto_trigger: True
 * /dvs_bootstrapping/enable_visualizations: True
 * /dvs_bootstrapping/events_scale_factor: 4.0
 * /dvs_bootstrapping/frame_size: 10000
 * /dvs_bootstrapping/local_frame_size: 5000
 * /dvs_bootstrapping/median_filter_size: 1
 * /dvs_bootstrapping/median_filtering: False
 * /dvs_bootstrapping/min_step_size: 5000
 * /dvs_bootstrapping/motion_corrected_topic: /events/image_raw
 * /dvs_bootstrapping/optical_flow_topic: /evo/bootstrap/op...
 * /dvs_bootstrapping/rate_hz: 30
 * /dvs_bootstrapping/unwarp_estimate_eps: 0.0001
 * /dvs_bootstrapping/unwarp_estimate_n_it: 75
 * /dvs_bootstrapping/unwarp_estimate_pyramid_lvls: 2
 * /dvs_frame_id: /dvs_evo
 * /dvs_mapping/accumulate_local_map_once_every: 10
 * /dvs_mapping/adaptive_threshold_c: 7
 * /dvs_mapping/adaptive_threshold_kernel_size: 5
 * /dvs_mapping/auto_trigger: False
 * /dvs_mapping/events_to_recreate_kf: 1000000
 * /dvs_mapping/frame_size: 4096
 * /dvs_mapping/global_point_cloud_skip_first: 5
 * /dvs_mapping/half_patchsize: 1
 * /dvs_mapping/median_filter_size: 15
 * /dvs_mapping/min_batch_size: 20000
 * /dvs_mapping/min_num_neighbors: 2
 * /dvs_mapping/min_num_neighbors_global_map: 2
 * /dvs_mapping/radius_search: 0.2
 * /dvs_mapping/radius_search_global_map: 0.05
 * /dvs_mapping/skip_batches: 0
 * /dvs_mapping/type_focus_measure: 0
 * /dvs_mapping/voxel_filter_leaf_size: 0.005
 * /dvs_reconstruction/init_cov: 10.0
 * /dvs_reconstruction/map_blur: 15
 * /dvs_reconstruction/sigma_m: 10.0
 * /dvs_reconstruction/window_size: 5000
 * /dvs_renderer_left/display_method: red-blue
 * /dvs_tracking/auto_trigger: False
 * /dvs_tracking/batch_size: 500
 * /dvs_tracking/discard_events_when_idle: True
 * /dvs_tracking/event_map_overlap_rate: 15
 * /dvs_tracking/events_per_kf: 100000
 * /dvs_tracking/frame_size: 5000
 * /dvs_tracking/map_blur: 3
 * /dvs_tracking/max_event_rate: 4000000
 * /dvs_tracking/max_iterations: 150
 * /dvs_tracking/min_map_size: 0
 * /dvs_tracking/min_n_keypoints: 0
 * /dvs_tracking/noise_rate: 10000
 * /dvs_tracking/pose_mean_filter_size: 5
 * /dvs_tracking/pyramid_levels: 2
 * /dvs_tracking/step_size: 15000
 * /fov_virtual_camera_deg: 80.0
 * /max_depth: 5
 * /min_depth: 0.4
 * /num_depth_cells: 100
 * /pose_to_tf/relative_to_first_pose: False
 * /pose_to_tf/source_topic_name: /svo/pose_cam/0
 * /rosdistro: melodic
 * /rosversion: 1.14.11
 * /snakify/length: 100
 * /svo/T_world_imuinit/qw: 1
 * /svo/T_world_imuinit/qx: 0
 * /svo/T_world_imuinit/qy: 0
 * /svo/T_world_imuinit/qz: 0
 * /svo/calib_file: /home/c4sp1/catki...
 * /svo/cam0_topic: /events/image_raw
 * /svo/depth_filter_affine_est_gain: False
 * /svo/depth_filter_affine_est_offset: True
 * /svo/detector_threshold_primary: 8
 * /svo/detector_threshold_secondary: 200
 * /svo/grid_size: 5
 * /svo/img_align_est_illumination_gain: False
 * /svo/img_align_est_illumination_offset: False
 * /svo/img_align_max_level: 4
 * /svo/img_align_min_level: 2
 * /svo/img_align_prior_lambda_rot: 0.0
 * /svo/img_align_prior_lambda_trans: 0.0
 * /svo/init_min_disparity: 12
 * /svo/kfselect_criterion: DOWNLOOKING
 * /svo/kfselect_min_angle: 20
 * /svo/kfselect_min_disparity: 30
 * /svo/kfselect_min_dist_metric: 0.1
 * /svo/kfselect_numkfs_lower_thresh: 70
 * /svo/kfselect_numkfs_upper_thresh: 120
 * /svo/map_scale: 0.6
 * /svo/max_depth_inv: 0.05
 * /svo/max_fts: 180
 * /svo/max_n_kfs: 30
 * /svo/mean_depth_inv: 0.3
 * /svo/min_depth_inv: 1.0
 * /svo/n_pyr_levels: 2
 * /svo/pipeline_is_stereo: False
 * /svo/poseoptim_prior_lambda: 0.0
 * /svo/poseoptim_thresh: 2.0
 * /svo/poseoptim_using_unit_sphere: False
 * /svo/publish_every_nth_dense_input: 5
 * /svo/publish_marker_scale: 0.5
 * /svo/reprojector_affine_est_gain: False
 * /svo/reprojector_affine_est_offset: True
 * /svo/reprojector_max_n_kfs: 5
 * /svo/runlc: False
 * /svo/scan_epi_unit_sphere: False
 * /svo/seed_convergence_sigma2_thresh: 100
 * /svo/update_seeds_with_old_keyframes: True
 * /svo/use_async_reprojectors: False
 * /svo/use_imu: False
 * /tf_to_camera_marker/marker_scale: 0.2
 * /trigger_map_expansion/baseline_threshold: 0.1
 * /trigger_map_expansion/coverage_threshold: 0.4
 * /trigger_map_expansion/number_of_initial_maps_to_skip: 0
 * /trigger_map_expansion/rate: 3
 * /trigger_map_expansion/visibility_threshold: 0.9
 * /virtual_height: 180
 * /virtual_width: 240
 * /world_frame_id: /world

NODES
  /
    dvs_bootstrapping (dvs_bootstrapping/dvs_bootstrapping_ef_ros)
    dvs_mapping (dvs_mapping/dvs_mapping_ros)
    dvs_reconstruction (dvs_reconstruction/dvs_reconstruction_ros)
    dvs_renderer_left (dvs_renderer/dvs_renderer)
    dvs_tracking (dvs_tracking/dvs_tracking_ros)
    pose_to_tf (dvs_bootstrapping/pose_to_tf.py)
    rqt_evo (rqt_evo/rqt_evo)
    rviz (rviz/rviz)
    snakify (evo_utils/snakify.py)
    svo (svo_ros/svo_node)
    svo_gui (rqt_gui/rqt_gui)
    tf_to_camera_marker (evo_utils/tf_to_camera_markers.py)
    trigger_map_expansion (dvs_mapping/trigger_map_expansion.py)

ROS_MASTER_URI=http://localhost:11311

process[svo-1]: started with pid [9955]
process[svo_gui-2]: started with pid [9956]
process[pose_to_tf-3]: started with pid [9957]
process[dvs_mapping-4]: started with pid [9958]
process[trigger_map_expansion-5]: started with pid [9959]
process[dvs_tracking-6]: started with pid [9960]
process[dvs_bootstrapping-7]: started with pid [9961]
process[dvs_reconstruction-8]: started with pid [9962]
process[tf_to_camera_marker-9]: started with pid [9963]
process[snakify-10]: started with pid [9964]
process[rqt_evo-11]: started with pid [9965]
process[dvs_renderer_left-12]: started with pid [9966]
[ INFO] [1628256826.469084173]: Found parameter: pipeline_is_stereo, value: 0
[ WARN] [1628256826.470455786]: Cannot find value for parameter: set_initial_attitude_from_gravity, assigning default: 1
[ WARN] [1628256826.470875246]: Cannot find value for parameter: automatic_reinitialization, assigning default: 0
[ INFO] [1628256826.471825962]: Found parameter: calib_file, value: /home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/ncamera/DAVIS-00000336.yaml
E20210806 15:33:46.472585  9955 yaml-serialization.h:42] Key "label" does not exist
E20210806 15:33:46.473126  9955 ncamera_yaml_serialization.cpp:23] Unable to get the label for the ncamera.
*** Aborted at 1628256826 (unix time) try "date -d @1628256826" if you are using GNU date ***
PC: @                0x0 (unknown)
*** SIGSEGV (@0x20) received by PID 9955 (TID 0x7f99e6ca5c00) from PID 32; stack trace: ***
    @     0x7f99e5b32040 (unknown)
    @     0x7f99e2ffada0 vk::cameras::NCamera::loadFromYaml()
    @     0x7f99e65585fa svo::factory::loadCameraFromYaml()
    @     0x7f99e6559f2c svo::factory::makeMono()
[ INFO] [1628256826.489966854]: Found parameter: auto_trigger, value: 1
    @     0x7f99e6539c72 svo::SvoInterface::SvoInterface()
I20210806 15:33:46.490939  9961 Bootstrapper.cpp:18] Bootstrapper initially idle: 0
[ INFO] [1628256826.492384926]: Found parameter: camera_name, value: DAVIS-00000336
    @     0x7f99e655ddc2 svo_ros::SvoNodeBase::SvoNodeBase()
    @     0x564864d5fccc main
[ INFO] [1628256826.493516099]: Found parameter: calib_file, value: /home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ INFO] [1628256826.494216692]: camera calibration URL: file:///home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ WARN] [1628256826.495146603]: Camera calibration file did not specify distortion model, assuming plumb bob
process[rviz-13]: started with pid [9971]
    @     0x7f99e5b14bf7 __libc_start_main
    @     0x564864d5fd6a _start
[ INFO] [1628256826.503788456]: Found parameter: world_frame_id, value: /world
[ INFO] [1628256826.504686048]: Found parameter: dvs_bootstrap_frame_id, value: /camera_0
[ INFO] [1628256826.525277681]: Found parameter: map_blur, value: 15
[ INFO] [1628256826.534749658]: Found parameter: window_size, value: 5000
[ INFO] [1628256826.537579360]: Found parameter: init_cov, value: 10
[ INFO] [1628256826.541735556]: Found parameter: sigma_m, value: 10
[ INFO] [1628256826.543117191]: Found parameter: unwarp_estimate_n_it, value: 75
[ INFO] [1628256826.544520312]: Found parameter: unwarp_estimate_eps, value: 0.0001
[ INFO] [1628256826.550520335]: Found parameter: unwarp_estimate_pyramid_lvls, value: 2
[ WARN] [1628256826.556308698]: Cannot find value for parameter: weight_scale_translation, assigning default: 0
[ WARN] [1628256826.557460908]: Cannot find value for parameter: weight_scale_rotation, assigning default: 0
[ INFO] [1628256826.558577450]: Found parameter: camera_name, value: DAVIS-00000336
I20210806 15:33:46.559587  9961 camera.cpp:21] Distortion type: plumb_bob
[ INFO] [1628256826.559739273]: Found parameter: calib_file, value: /home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ INFO] [1628256826.559771871]: Found parameter: camera_name, value: DAVIS-00000336
[ INFO] [1628256826.560892044]: camera calibration URL: file:///home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ WARN] [1628256826.561687646]: Camera calibration file did not specify distortion model, assuming plumb bob
I20210806 15:33:46.562676  9960 tracker.cpp:58] Field of view: 58.3228
[ INFO] [1628256826.563946986]: Found parameter: calib_file, value: /home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ INFO] [1628256826.564912608]: camera calibration URL: file:///home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ WARN] [1628256826.568286699]: Camera calibration file did not specify distortion model, assuming plumb bob
[ INFO] [1628256826.580977640]: Found parameter: dvs_frame_id, value: /dvs_evo
[ INFO] [1628256826.582692561]: Found parameter: world_frame_id, value: /world
[ INFO] [1628256826.583439660]: Found parameter: rate_hz, value: 30
[ INFO] [1628256826.584685046]: Found parameter: frame_size, value: 10000
[ INFO] [1628256826.585244927]: Found parameter: camera_name, value: DAVIS-00000336
[ INFO] [1628256826.586043295]: Found parameter: local_frame_size, value: 5000
[ INFO] [1628256826.587934374]: Found parameter: calib_file, value: /home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ INFO] [1628256826.589511207]: Found parameter: enable_visualizations, value: 1
[ INFO] [1628256826.589628735]: camera calibration URL: file:///home/c4sp1/catkin_ws/src/rpg_dvs_evo_open/dvs_tracking/parameters/calib/DAVIS-00000336.yaml
[ WARN] [1628256826.590702738]: Camera calibration file did not specify distortion model, assuming plumb bob
[ INFO] [1628256826.591994955]: Found parameter: motion_corrected_topic, value: /events/image_raw
[ INFO] [1628256826.610704396]: Found parameter: dvs_frame_id, value: /dvs_evo
[ INFO] [1628256826.613024038]: Found parameter: world_frame_id, value: /world
[ INFO] [1628256826.617853193]: Found parameter: auto_trigger, value: 0
I20210806 15:33:46.617935 10078 tracker.cpp:150] Spawned tracking thread.
[ INFO] [1628256826.633172076]: Found parameter: radius_search, value: 0.2
[ INFO] [1628256826.633962877]: Found parameter: min_num_neighbors, value: 2
[ INFO] [1628256826.634822015]: Found parameter: median_filter_size, value: 15
[ INFO] [1628256826.638284844]: Found parameter: dvs_frame_id, value: /dvs_evo
[ INFO] [1628256826.643086924]: Found parameter: dvs_bootstrap_frame_id, value: /camera_0
[ INFO] [1628256826.645403596]: Found parameter: world_frame_id, value: /world
[ INFO] [1628256826.663143296]: Found parameter: optical_flow_topic, value: /evo/bootstrap/optical_flow
[ INFO] [1628256826.665139472]: Found parameter: events_to_recreate_kf, value: 1000000
[ INFO] [1628256826.679660001]: Found parameter: fov_virtual_camera_deg, value: 80
[ INFO] [1628256826.680953095]: Found parameter: virtual_width, value: 240
[ INFO] [1628256826.681975992]: Found parameter: virtual_height, value: 180
I20210806 15:33:46.682035  9958 depth_defocus_node.cpp:63] Focal length of virtual camera: 143.01 pixels
[ INFO] [1628256826.746015266]: Found parameter: discard_events_when_idle, value: 1
[ INFO] [1628256826.754285933]: Found parameter: dvs_frame_id, value: /dvs_evo
[ INFO] [1628256826.766214629]: Found parameter: world_frame_id, value: /world
[ INFO] [1628256826.767521354]: Found parameter: min_step_size, value: 5000
[ INFO] [1628256826.770098893]: Found parameter: events_scale_factor, value: 4
[ INFO] [1628256826.771820953]: Found parameter: activation_threshold_min, value: 10
[svo-1] process has died [pid 9955, exit code -11, cmd /home/c4sp1/catkin_ws/devel/lib/svo_ros/svo_node events:=/dvs/events remote_key:=/evo/remote_key __name:=svo __log:=/home/c4sp1/.ros/log/a996bfae-f69f-11eb-b50f-b07b25408b14/svo-1.log].
log file: /home/c4sp1/.ros/log/a996bfae-f69f-11eb-b50f-b07b25408b14/svo-1*.log
[ INFO] [1628256826.773273418]: Found parameter: activation_threshold_patch_size, value: 13
[ INFO] [1628256826.774985234]: Found parameter: median_filter_size, value: 1
[ INFO] [1628256826.776230894]: Found parameter: median_filtering, value: 0
[ INFO] [1628256826.777452838]: Found parameter: adaptive_thresholding, value: 1
[ INFO] [1628256826.805999266]: Found parameter: min_depth, value: 0.4
[ INFO] [1628256826.806657670]: Found parameter: max_depth, value: 5
[ INFO] [1628256826.807303963]: Found parameter: num_depth_cells, value: 100
[ INFO] [1628256826.808142413]: Found parameter: adaptive_threshold_kernel_size, value: 5
[ INFO] [1628256826.808920041]: Found parameter: adaptive_threshold_c, value: 7
[ INFO] [1628256826.815296859]: Found parameter: auto_trigger, value: 0
[ WARN] [1628256826.954421017]: Cannot find value for parameter: max_events_frames_saved_to_file, assigning default: 0
[ WARN] [1628256826.954820852]: Cannot find value for parameter: events_frames_filename_format, assigning default: /media/user/data/example/%06d.bmp
OpenCV Error: Iterations do not converge (The algorithm stopped before its convergence. The correlation is going to be minimized. Images may be uncorrelated or non-overlapped) in findTransformECC, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/video/src/ecc.cpp, line 530
W20210806 15:33:55.505833 10107 motion_correction.cpp:69] The algorithm stopped before its convergence. The correlation is going to be minimized. Images may be uncorrelated or non-overlapped
OpenCV Error: Iterations do not converge (The algorithm stopped before its convergence. The correlation is going to be minimized. Images may be uncorrelated or non-overlapped) in findTransformECC, file /build/opencv-L2vuMj/opencv-3.2.0+dfsg/modules/video/src/ecc.cpp, line 530
W20210806 15:33:59.092280 10107 motion_correction.cpp:69] The algorithm stopped before its convergence. The correlation is going to be minimized. Images may be uncorrelated or non-overlapped
^C[rviz-13] killing on exit
[dvs_renderer_left-12] killing on exit
[rqt_evo-11] killing on exit
[snakify-10] killing on exit
[tf_to_camera_marker-9] killing on exit
[dvs_reconstruction-8] killing on exit
[dvs_tracking-6] killing on exit
[dvs_bootstrapping-7] killing on exit
[trigger_map_expansion-5] killing on exit
[dvs_mapping-4] killing on exit
[pose_to_tf-3] killing on exit
[svo_gui-2] killing on exit
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
*** Aborted at 1628256843 (unix time) try "date -d @1628256843" if you are using GNU date ***
[pcl::KdTreeFLANN::setInputCloud] Cannot create a KDTree with an empty input cloud!
shutting down processing monitor...
... shutting down processing monitor complete
done
YoungHaulk commented 1 year ago

I ran the live mode with davis346 and I had the same problem as you. This problem is very troublesome How did you solve it?