Needed help to start H2-Mapping for realsense camera live data. Getting some error

ShreyasKuntnal commented 3 months ago

I setup everthing according to the setup but failed find right tutorial or steps to launch but tried with these steps.

roscore
roslaunch for realsense camera
roslaunch for vins estimator

the conda activate h2mapping and later ran this comand: python3 -W ignore demo/run_mapping.py configs/realsense/realsense.yaml -run_ros this gives output:

Traceback (most recent call last):
File "demo/run_mapping.py", line 25, in <module>
slam = H2Mapping(args)
File "/home/vt008/H2-Mapping/mapping/src/H2Mapping.py", line 10, in __init__
self.logger = BasicLogger(args)
File "/home/vt008/H2-Mapping/mapping/src/loggers.py", line 20, in __init__
args.log_dir, args.exp_name, self.get_random_time_str())
AttributeError: 'Namespace' object has no attribute 'exp_name'

So, Can you please help me on what went wrong here?

JIANG-CX commented 3 months ago

Hi, based on the step 5 in “self-captured house dataset", you should use "

configs/realsense/tower.yaml".

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Shreyas K @.> 发送时间: Sunday, June 30, 2024 3:05:14 AM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: Subscribed @.***> 主题: [SYSU-STAR/H2-Mapping] Needed help to start H2-Mapping for realsense camera live data. Getting some error (Issue #38)

I setup everthing according to the setup but failed find right tutorial or steps to launch but tried with these steps.

roscore
roslaunch for realsense camera
roslaunch for vins estimator
the conda activate h2mapping and later ran this comand: python3 -W ignore demo/run_mapping.py configs/realsense/realsense.yaml -run_ros this gives output:

Traceback (most recent call last): File "demo/run_mapping.py", line 25, in slam = H2Mapping(args) File "/home/vt008/H2-Mapping/mapping/src/H2Mapping.py", line 10, in init self.logger = BasicLogger(args) File "/home/vt008/H2-Mapping/mapping/src/loggers.py", line 20, in init args.log_dir, args.exp_name, self.get_random_time_str()) AttributeError: 'Namespace' object has no attribute 'exp_name'

So, Can you please help me on what went wrong here?

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/38, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNQRR3E2RRSODXHCIJTZJ4AOVAVCNFSM6AAAAABKDNHS4SVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM4DCOJSGYZTSMI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

ShreyasKuntnal commented 3 months ago

I guess I wasn't clear or you are not getting what I need. But Here if i used Tower.yaml will I be able to record mapping of my own house with a realsense camera?

Tower.yaml is only for the dataset bag file which we will download from onedrive link right?

If I need to create a mapping or 3d ply mesh of my home how to do is my question.

I just wanna run the algorithm on laptop with realsense camera connected and create mapping and save the 3d Ply file.

JIANG-CX commented 3 months ago

You can follow the guidance in "Use your own rgbd sequence". You should first modify the yaml（check both realsense.yaml and tower.yaml）based on the guidance. Then you launch the realsense camera and vins to verify the tracking module is successfully running. Then run conda activate h2mapping cd H2-Mapping source devel/setup.bash cd mapping python -W ignore demo/run_mapping.py configs/realsense/tower.yaml -run_ros to start mapping

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Shreyas K @.> 发送时间: Sunday, June 30, 2024 11:41:23 AM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] Needed help to start H2-Mapping for realsense camera live data. Getting some error (Issue #38)

I guess I wasn't clear or I am not getting what I need. But Here if i used Tower.yaml will I be able to record mapping of my own house with a realsense camera?

Tower.yaml is only for the dataset bag file which we will download from onedrive linka right?

If I need to create a mapping or 3d ply mesh of my home how to do is my question.

I just wanna run the algorithm on laptop with realsense camera connected and create mapping and save the 3d Ply file.

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/38#issuecomment-2198421922, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNWOJHXB4GZH7BPYRM3ZJ546HAVCNFSM6AAAAABKDNHS4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJYGQZDCOJSGI. You are receiving this because you commented.Message ID: @.***>

JIANG-CX commented 3 months ago

It is suitable for both the recorded bag or online data stream.

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: JIANG Chenxing @.> 发送时间: Sunday, June 30, 2024 11:49:34 AM 收件人: SYSU-STAR/H2-Mapping @.>; SYSU-STAR/H2-Mapping @.> 抄送: Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] Needed help to start H2-Mapping for realsense camera live data. Getting some error (Issue #38)

You can follow the guidance in "Use your own rgbd sequence". You should first modify the yaml（check both realsense.yaml and tower.yaml）based on the guidance. Then you launch the realsense camera and vins to verify the tracking module is successfully running. Then run conda activate h2mapping cd H2-Mapping source devel/setup.bash cd mapping python -W ignore demo/run_mapping.py configs/realsense/tower.yaml -run_ros to start mapping

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Shreyas K @.> 发送时间: Sunday, June 30, 2024 11:41:23 AM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] Needed help to start H2-Mapping for realsense camera live data. Getting some error (Issue #38)

I guess I wasn't clear or I am not getting what I need. But Here if i used Tower.yaml will I be able to record mapping of my own house with a realsense camera?

Tower.yaml is only for the dataset bag file which we will download from onedrive linka right?

If I need to create a mapping or 3d ply mesh of my home how to do is my question.

I just wanna run the algorithm on laptop with realsense camera connected and create mapping and save the 3d Ply file.

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/38#issuecomment-2198421922, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNWOJHXB4GZH7BPYRM3ZJ546HAVCNFSM6AAAAABKDNHS4SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJYGQZDCOJSGI. You are receiving this because you commented.Message ID: @.***>

ShreyasKuntnal commented 3 months ago

Oh ok will check about that and update back about it meanwhile, So may I know how to fix that particular error when we run realsense.yaml file?

Traceback (most recent call last):
  File "demo/run_mapping.py", line 25, in <module>
    slam = H2Mapping(args)
  File "/home/vt008/H2-Mapping/mapping/src/H2Mapping.py", line 10, in __init__
    self.logger = BasicLogger(args)
  File "/home/vt008/H2-Mapping/mapping/src/loggers.py", line 20, in __init__
    args.log_dir, args.exp_name, self.get_random_time_str())
AttributeError: 'Namespace' object has no attribute 'exp_name'

JIANG-CX commented 3 months ago

The realsense.yaml config serves as the base configuration for tower.yaml, like replica.yaml is the base configuration for room.yaml. When you use tower.yaml, it will inherit the content from realsense.yaml, so you should not directly use realsense.yaml.

ShreyasKuntnal commented 3 months ago

So Tried above things but .ply is not generated correctly, Neither I can understand what needs to be done. Here are the outputs:

Roslaunch for realsense:

SUMMARY
========

PARAMETERS
 * /camera/l500_depth_sensor/global_time_enabled: True
 * /camera/motion_module/global_time_enabled: True
 * /camera/realsense2_camera/accel_fps: 250
 * /camera/realsense2_camera/accel_frame_id: camera_accel_frame
 * /camera/realsense2_camera/accel_optical_frame_id: camera_accel_opti...
 * /camera/realsense2_camera/align_depth: True
 * /camera/realsense2_camera/aligned_depth_to_color_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_fisheye_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra1_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/aligned_depth_to_infra2_frame_id: camera_aligned_de...
 * /camera/realsense2_camera/allow_no_texture_points: False
 * /camera/realsense2_camera/base_frame_id: camera_link
 * /camera/realsense2_camera/calib_odom_file: 
 * /camera/realsense2_camera/clip_distance: -2.0
 * /camera/realsense2_camera/color_fps: 30
 * /camera/realsense2_camera/color_frame_id: camera_color_frame
 * /camera/realsense2_camera/color_height: 480
 * /camera/realsense2_camera/color_optical_frame_id: camera_color_opti...
 * /camera/realsense2_camera/color_width: 640
 * /camera/realsense2_camera/confidence_fps: -1
 * /camera/realsense2_camera/confidence_height: -1
 * /camera/realsense2_camera/confidence_width: -1
 * /camera/realsense2_camera/depth_fps: 30
 * /camera/realsense2_camera/depth_frame_id: camera_depth_frame
 * /camera/realsense2_camera/depth_height: 480
 * /camera/realsense2_camera/depth_optical_frame_id: camera_depth_opti...
 * /camera/realsense2_camera/depth_width: 640
 * /camera/realsense2_camera/device_type: 
 * /camera/realsense2_camera/enable_accel: True
 * /camera/realsense2_camera/enable_color: True
 * /camera/realsense2_camera/enable_confidence: True
 * /camera/realsense2_camera/enable_depth: True
 * /camera/realsense2_camera/enable_fisheye1: False
 * /camera/realsense2_camera/enable_fisheye2: False
 * /camera/realsense2_camera/enable_fisheye: False
 * /camera/realsense2_camera/enable_gyro: True
 * /camera/realsense2_camera/enable_infra1: True
 * /camera/realsense2_camera/enable_infra2: True
 * /camera/realsense2_camera/enable_infra: True
 * /camera/realsense2_camera/enable_pointcloud: False
 * /camera/realsense2_camera/enable_pose: False
 * /camera/realsense2_camera/enable_sync: True
 * /camera/realsense2_camera/filters: 
 * /camera/realsense2_camera/fisheye1_frame_id: camera_fisheye1_f...
 * /camera/realsense2_camera/fisheye1_optical_frame_id: camera_fisheye1_o...
 * /camera/realsense2_camera/fisheye2_frame_id: camera_fisheye2_f...
 * /camera/realsense2_camera/fisheye2_optical_frame_id: camera_fisheye2_o...
 * /camera/realsense2_camera/fisheye_fps: -1
 * /camera/realsense2_camera/fisheye_frame_id: camera_fisheye_frame
 * /camera/realsense2_camera/fisheye_height: -1
 * /camera/realsense2_camera/fisheye_optical_frame_id: camera_fisheye_op...
 * /camera/realsense2_camera/fisheye_width: -1
 * /camera/realsense2_camera/gyro_fps: 200
 * /camera/realsense2_camera/gyro_frame_id: camera_gyro_frame
 * /camera/realsense2_camera/gyro_optical_frame_id: camera_gyro_optic...
 * /camera/realsense2_camera/imu_optical_frame_id: camera_imu_optica...
 * /camera/realsense2_camera/infra1_frame_id: camera_infra1_frame
 * /camera/realsense2_camera/infra1_optical_frame_id: camera_infra1_opt...
 * /camera/realsense2_camera/infra2_frame_id: camera_infra2_frame
 * /camera/realsense2_camera/infra2_optical_frame_id: camera_infra2_opt...
 * /camera/realsense2_camera/infra_fps: 30
 * /camera/realsense2_camera/infra_height: 480
 * /camera/realsense2_camera/infra_rgb: False
 * /camera/realsense2_camera/infra_width: 640
 * /camera/realsense2_camera/initial_reset: True
 * /camera/realsense2_camera/json_file_path: 
 * /camera/realsense2_camera/linear_accel_cov: 0.01
 * /camera/realsense2_camera/odom_frame_id: camera_odom_frame
 * /camera/realsense2_camera/ordered_pc: False
 * /camera/realsense2_camera/pointcloud_texture_index: 0
 * /camera/realsense2_camera/pointcloud_texture_stream: RS2_STREAM_COLOR
 * /camera/realsense2_camera/pose_frame_id: camera_pose_frame
 * /camera/realsense2_camera/pose_optical_frame_id: camera_pose_optic...
 * /camera/realsense2_camera/publish_odom_tf: True
 * /camera/realsense2_camera/publish_tf: True
 * /camera/realsense2_camera/reconnect_timeout: 6.0
 * /camera/realsense2_camera/rosbag_filename: 
 * /camera/realsense2_camera/serial_no: 
 * /camera/realsense2_camera/stereo_module/exposure/1: 7500
 * /camera/realsense2_camera/stereo_module/exposure/2: 1
 * /camera/realsense2_camera/stereo_module/gain/1: 16
 * /camera/realsense2_camera/stereo_module/gain/2: 16
 * /camera/realsense2_camera/tf_publish_rate: 0.0
 * /camera/realsense2_camera/topic_odom_in: odom_in
 * /camera/realsense2_camera/unite_imu_method: linear_interpolation
 * /camera/realsense2_camera/usb_port_id: 
 * /camera/realsense2_camera/wait_for_device_timeout: -1.0
 * /camera/rgb_camera/global_time_enabled: True
 * /camera/stereo_module/emitter_enabled: 0
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /camera/
    realsense2_camera (nodelet/nodelet)
    realsense2_camera_manager (nodelet/nodelet)

ROS_MASTER_URI=http://localhost:11311

process[camera/realsense2_camera_manager-1]: started with pid [57334]
process[camera/realsense2_camera-2]: started with pid [57335]
[ INFO] [1719813914.555784976]: Initializing nodelet with 16 worker threads.
[ INFO] [1719813914.625014727]: RealSense ROS v2.3.2
[ INFO] [1719813914.625035855]: Built with LibRealSense v2.50.0
[ INFO] [1719813914.625043112]: Running with LibRealSense v2.50.0
[ INFO] [1719813914.653071464]:  
[ INFO] [1719813914.945300638]: Device with serial number 243522070607 was found.

[ INFO] [1719813914.945354269]: Device with physical ID 4-4-4 was found.
[ INFO] [1719813914.945369416]: Device with name Intel RealSense D435I was found.
[ INFO] [1719813914.945919142]: Device with port number 4-4 was found.
[ INFO] [1719813914.945954546]: Device USB type: 3.2
[ INFO] [1719813914.945988524]: Resetting device...
[ INFO] [1719813921.050516835]:  
[ INFO] [1719813921.259702715]: Device with serial number 243522070607 was found.

[ INFO] [1719813921.259764000]: Device with physical ID 4-4-5 was found.
[ INFO] [1719813921.259795521]: Device with name Intel RealSense D435I was found.
[ INFO] [1719813921.260371350]: Device with port number 4-4 was found.
[ INFO] [1719813921.260401528]: Device USB type: 3.2
[ INFO] [1719813921.262530651]: getParameters...
[ INFO] [1719813921.285323997]: setupDevice...
[ INFO] [1719813921.285352750]: JSON file is not provided
[ INFO] [1719813921.285368124]: ROS Node Namespace: camera
[ INFO] [1719813921.285385016]: Device Name: Intel RealSense D435I
[ INFO] [1719813921.285399471]: Device Serial No: 243522070607
[ INFO] [1719813921.285414346]: Device physical port: 4-4-5
[ INFO] [1719813921.285427831]: Device FW version: 05.13.00.50
[ INFO] [1719813921.285442121]: Device Product ID: 0x0B3A
[ INFO] [1719813921.285457253]: Enable PointCloud: Off
[ INFO] [1719813921.285470903]: Align Depth: On
[ INFO] [1719813921.285485973]: Sync Mode: On
[ INFO] [1719813921.285533127]: Device Sensors: 
[ INFO] [1719813921.370895320]: Stereo Module was found.
[ INFO] [1719813921.380570999]: RGB Camera was found.
[ INFO] [1719813921.380679603]: Motion Module was found.
[ INFO] [1719813921.380705846]: (Infrared, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1719813921.380718519]: (Confidence, 0) sensor isn't supported by current device! -- Skipping...
[ INFO] [1719813921.380919211]: num_filters: 1
[ INFO] [1719813921.380931892]: Setting Dynamic reconfig parameters.
[ INFO] [1719813924.859164840]: Done Setting Dynamic reconfig parameters.
[ INFO] [1719813924.859720874]: depth stream is enabled - width: 640, height: 480, fps: 30, Format: Z16
[ INFO] [1719813924.859947167]: infra1 stream is enabled - width: 640, height: 480, fps: 30, Format: Y8
[ INFO] [1719813924.860174595]: infra2 stream is enabled - width: 640, height: 480, fps: 30, Format: Y8
[ INFO] [1719813924.860696211]: color stream is enabled - width: 640, height: 480, fps: 30, Format: RGB8
[ INFO] [1719813924.863087565]: gyro stream is enabled - fps: 200
[ WARN] [1719813924.863115716]: No mathcing profile found for accel with fps=250
[ WARN] [1719813924.863130101]: Using default profile instead.
[ INFO] [1719813924.863142452]: accel stream is enabled - fps: 100
[ INFO] [1719813924.863159445]: setupPublishers...
[ INFO] [1719813924.864973225]: Expected frequency for depth = 30.00000
[ INFO] [1719813924.883283122]: Expected frequency for infra1 = 30.00000
[ INFO] [1719813924.898426964]: Expected frequency for infra2 = 30.00000
[ INFO] [1719813924.911616332]: Expected frequency for color = 30.00000
[ INFO] [1719813924.928764098]: Expected frequency for aligned_depth_to_color = 30.00000
[ INFO] [1719813924.945187242]: Start publisher IMU
[ INFO] [1719813924.947313089]: setupStreams...
 01/07 11:35:25,163 WARNING [139876364363520] (ds5-motion.cpp:473) IMU Calibration is not available, default intrinsic and extrinsic will be used.
[ WARN] [1719813925.435076396]: 
[ WARN] [1719813925.435125901]: frame's time domain is HARDWARE_CLOCK. Timestamps may reset periodically.
 01/07 11:35:25,537 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
[ INFO] [1719813925.587674947]: SELECTED BASE:Depth, 0
[ INFO] [1719813925.597268542]: RealSense Node Is Up!
 01/07 11:35:25,659 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:25,709 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:25,760 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:25,896 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:25,946 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:25,997 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,048 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,098 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,149 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,199 ERROR [139875561682688] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 130
 01/07 11:35:26,200 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,250 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,301 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,352 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,402 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,453 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,504 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,535 ERROR [139875063949056] (uvc-streamer.cpp:106) uvc streamer watchdog triggered on endpoint: 131
 01/07 11:35:26,554 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,605 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,656 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,706 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,757 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,808 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,859 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,909 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:26,960 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:27,011 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:27,062 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:27,415 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:27,566 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:27,617 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
[ WARN] [1719813927.860944452]: Hardware Notification:Depth stream start failure,1.71981e+12,Error,Hardware Error
 01/07 11:35:51,386 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:35:51,437 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Success, number: 0
 01/07 11:36:07,498 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:07,549 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:10,601 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:10,651 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:11,703 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:11,753 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:13,805 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:13,855 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:14,907 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:14,958 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:16,009 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:16,060 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:16,855 ERROR [139875038770944] (ds5-options.cpp:88) Asic Temperature value is not valid!
 01/07 11:36:17,112 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:17,162 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:17,861 ERROR [139875038770944] (ds5-options.cpp:88) Asic Temperature value is not valid!
 01/07 11:36:18,866 ERROR [139875038770944] (ds5-options.cpp:88) Asic Temperature value is not valid!
 01/07 11:36:29,224 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 768, error: Resource temporarily unavailable, number: 11
 01/07 11:36:29,275 WARNING [139875927971584] (messenger-libusb.cpp:42) control_transfer returned error, index: 300, error: Resource temporarily unavailable, number: b

2.roslaunch for vins estimator: roslaunch vins_estimator uav_nerf.launch

SUMMARY
========

PARAMETERS
 * /pose_graph/config_file: /home/vt008/H2-Ma...
 * /pose_graph/skip_cnt: 0
 * /pose_graph/skip_dis: 0.0
 * /pose_graph/visualization_shift_x: 0
 * /pose_graph/visualization_shift_y: 0
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /vins_estimator/config_file: /home/vt008/H2-Ma...
 * /vins_estimator/vins_folder: /home/vt008/H2-Ma...

NODES
  /
    pose_graph (pose_graph/pose_graph)
    rvizvisualisation (rviz/rviz)
    vins_estimator (vins_estimator/vins_estimator)

ROS_MASTER_URI=http://localhost:11311

process[vins_estimator-1]: started with pid [57940]
process[pose_graph-2]: started with pid [57942]
process[rvizvisualisation-3]: started with pid [57944]
[ INFO] [1719813924.754215905]: init begins
[ INFO] [1719813924.761615628]: Loaded config_file: /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/uav2022/uav_nerf.yaml
[ INFO] [1719813924.762486414]: Loaded vins_folder: /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/../
result path /home/jcx/Documents/RAL2023/ros_ws//vins_result_no_loop.csv
/home/jcx/Documents/RAL2023/ros_ws/ not exists, trying to create it 
/home/jcx/Documents/RAL2023 not exists, trying to create it 
/home/jcx/Documents not exists, trying to create it 
/home/jcx not exists, trying to create it 
Failed to create folder /home/jcx 
[ INFO] [1719813924.762542600]: ROW: 480 COL: 640 
[ WARN] [1719813924.762551037]:  fix extrinsic param 
[ INFO] [1719813924.762600718]: Extrinsic_R : 
   0.999969  0.00614047  0.00501018
-0.00594551    0.999258  -0.0380416
-0.00524006   0.0380106    0.999264
[ INFO] [1719813924.762612683]: Extrinsic_T : 
-0.000121045    -0.020129   -0.0219023
[ INFO] [1719813924.762622135]: Synchronized sensors, fix time offset: -0.015
[ INFO] [1719813924.762639383]: reading paramerter of camera /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/uav2022/uav_nerf.yaml
[ WARN] [1719813924.762775138]: Waiting for image and imu...
[ WARN] [1719813927.241230076]: First_flag
[ INFO] [1719813928.537778236]: IMU excitation not enouth!
[ INFO] [1719813928.744881006]: IMU excitation not enouth!
[ INFO] [1719813928.875399086]: IMU excitation not enouth!
[ INFO] [1719813929.075426840]: IMU excitation not enouth!
[ INFO] [1719813929.211178172]: IMU excitation not enouth!
[ INFO] [1719813929.406136991]: IMU excitation not enouth!
[ INFO] [1719813929.511031873]: IMU excitation not enouth!
[ INFO] [1719813929.611424035]: IMU excitation not enouth!
[ INFO] [1719813929.710832095]: IMU excitation not enouth!
[ INFO] [1719813929.806214423]: IMU excitation not enouth!
[ INFO] [1719813929.905201672]: IMU excitation not enouth!
[ INFO] [1719813930.011536587]: IMU excitation not enouth!
[ INFO] [1719813930.112027788]: IMU excitation not enouth!
[ INFO] [1719813930.214214732]: IMU excitation not enouth!
[ INFO] [1719813930.306129857]: IMU excitation not enouth!
[ INFO] [1719813930.408962429]: IMU excitation not enouth!
[ INFO] [1719813930.614421066]: IMU excitation not enouth!
[ WARN] [1719813930.813154445]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813223038]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813237052]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813249446]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813276654]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813295579]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813331728]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813355288]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813380346]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813930.813405729]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813930.813432866]: Not enough features or parallax; Move device around
[ WARN] [1719813931.009867682]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.009918962]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.009935050]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.009964531]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.009983002]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.010009060]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.010024384]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.010040860]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.010063973]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.010093988]: [solveTwoViewGeometry] average parallex: 21.665152
[ INFO] [1719813931.010110900]: Not enough features or parallax; Move device around
[ WARN] [1719813931.212980470]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213023761]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213041411]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213051688]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213062474]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213074311]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213085857]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213097240]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213108018]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.213120107]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813931.213130751]: Not enough features or parallax; Move device around
[ WARN] [1719813931.409989238]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410073706]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410094166]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410127232]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410149432]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410169059]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410187516]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410210125]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410230113]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.410250320]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813931.410269179]: Not enough features or parallax; Move device around
[ WARN] [1719813931.580150913]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580239889]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580260805]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580292277]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580311064]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580340071]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580363735]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580394967]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580413762]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.580433650]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813931.580451026]: Not enough features or parallax; Move device around
[ WARN] [1719813931.778877586]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778922353]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778933572]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778943983]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778956383]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778968058]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778979274]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.778990843]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.779002388]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.779019846]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813931.779041996]: Not enough features or parallax; Move device around
[ WARN] [1719813931.979648937]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979703879]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979722237]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979739052]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979754980]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979771090]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979799913]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979821335]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979836779]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813931.979851889]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813931.979866132]: Not enough features or parallax; Move device around
[ WARN] [1719813932.179706895]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179760828]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179775357]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179788936]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179818132]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179834863]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179848873]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179872405]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179888366]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.179903356]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813932.179932488]: Not enough features or parallax; Move device around
[ WARN] [1719813932.378196692]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378240809]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378254599]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378269645]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378284854]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378309285]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378323931]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378337937]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378353121]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.378367579]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719813932.378380801]: Not enough features or parallax; Move device around
[ WARN] [1719813932.545476321]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545526211]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545543315]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545566321]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545579168]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545591799]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545604018]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.545616775]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.629630
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
[ WARN] [1719813932.744790872]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744872988]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744893326]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744920924]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744940702]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744963332]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.744984302]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.745008643]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.909091
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
[ WARN] [1719813932.950802186]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950834837]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950843575]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950851608]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950859212]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950866806]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813932.950874582]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.913043
unstable features tracking, please slowly move you device!
[ WARN] [1719813933.145996373]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.146102723]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.146120615]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.146139318]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.146160529]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.909091
unstable features tracking, please slowly move you device!
[ WARN] [1719813933.312605724]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.312663762]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719813933.312693394]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.818182
[GlobalSFM::evaluateError] ATE P: 0.002611
[GlobalSFM::evaluateError] ATE D: 0.206356
[GlobalSFM::evaluateError] ATE P: 0.001910
[GlobalSFM::evaluateError] ATE D: 0.126606
Ceres Solver Report: Iterations: 10, Initial cost: 1.593611e+03, Final cost: 5.815530e+02, Termination: CONVERGENCE
[ WARN] [1719813933.327677852]: gyroscope bias initial calibration -0.00827174   0.0151728  -0.0494096
[ INFO] [1719813933.328757491]:  result g     9.67403 -0.440887  -9.55561   1.44318
[ INFO] [1719813933.328838503]:  refine     9.805 -0.446856  -9.68498   1.46272
R0 from g:        0  2.61212 -81.4116
[ INFO] [1719813933.330040161]: g0     -6.93889e-17  2.22045e-16        9.805
[ INFO] [1719813933.330064305]: my R0  1.49544e-16    -4.47214    -95.1414
[ INFO] [1719813933.354268985]: Initialization finish!
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.2.0) ../modules/video/src/lkpyramid.cpp:1257: error: (-215:Assertion failed) (npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function 'calc'

[vins_estimator-1] process has died [pid 57940, exit code -6, cmd /home/vt008/H2-Mapping/devel/lib/vins_estimator/vins_estimator __name:=vins_estimator __log:=/home/vt008/.ros/log/b022f3e2-3760-11ef-89f9-919c5294b83d/vins_estimator-1.log].
log file: /home/vt008/.ros/log/b022f3e2-3760-11ef-89f9-919c5294b83d/vins_estimator-1*.log

So, Can you please help me debug whats wrong, or else how to fix these issues?

JIANG-CX commented 3 months ago

It seems that the tracking module has failed. You can try our provided bag first to make sure the tracking module is installed correctly.

ShreyasKuntnal commented 3 months ago

Oh ok, I kept bag file for decompression, Meanwhile, there are no any other issues in realsense camera nor the vins estimator output that I can work on fixing?

Extremely sorry for the multiple questions on this thing. I am bit new to these things so, Your replies will help me a lot in exploring even more in this field.

JIANG-CX commented 3 months ago

For RealSense, you can use rqt_image_view to verify whether you have successfully received the RGB and depth images, or you can check the data stream using ROS topics.

Regarding VINS, it appears that the system fails after initialization, but I cannot determine the source of the error based on the console output. You can send me a screenshot or other details, or you can refer to the numerous tutorials available for running VINS-Mono or VINS-Fusion. The main difference in our tracking module is the use of depth images.

ShreyasKuntnal commented 3 months ago

I have checked on rqt_image_view stream is evident there, and the rostopic are publishing data. But why does dvins stop with red mark and even mapping process too stops.

Terminal outputs: 1.bash ros_cmd/run_vins_rgbd.sh :

SUMMARY
========

PARAMETERS
 * /pose_graph/config_file: /home/vt008/H2-Ma...
 * /pose_graph/skip_cnt: 0
 * /pose_graph/skip_dis: 0.0
 * /pose_graph/visualization_shift_x: 0
 * /pose_graph/visualization_shift_y: 0
 * /rosdistro: noetic
 * /rosversion: 1.16.0
 * /vins_estimator/config_file: /home/vt008/H2-Ma...
 * /vins_estimator/vins_folder: /home/vt008/H2-Ma...

NODES
  /
    pose_graph (pose_graph/pose_graph)
    rvizvisualisation (rviz/rviz)
    vins_estimator (vins_estimator/vins_estimator)

ROS_MASTER_URI=http://localhost:11311

process[vins_estimator-1]: started with pid [13082]
process[pose_graph-2]: started with pid [13083]
process[rvizvisualisation-3]: started with pid [13084]
[ INFO] [1719826597.633981517]: init begins
[ INFO] [1719826597.642896800]: Loaded config_file: /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/uav2022/uav_nerf.yaml
[ INFO] [1719826597.644509147]: Loaded vins_folder: /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/../
result path /home/jcx/Documents/RAL2023/ros_ws//vins_result_no_loop.csv
/home/jcx/Documents/RAL2023/ros_ws/ not exists, trying to create it 
/home/jcx/Documents/RAL2023 not exists, trying to create it 
/home/jcx/Documents not exists, trying to create it 
/home/jcx not exists, trying to create it 
Failed to create folder /home/jcx 
[ INFO] [1719826597.644564744]: ROW: 480 COL: 640 
[ WARN] [1719826597.644574163]:  fix extrinsic param 
[ INFO] [1719826597.644622850]: Extrinsic_R : 
   0.999969  0.00614047  0.00501018
-0.00594551    0.999258  -0.0380416
-0.00524006   0.0380106    0.999264
[ INFO] [1719826597.644644404]: Extrinsic_T : 
-0.000121045    -0.020129   -0.0219023
[ INFO] [1719826597.644660286]: Synchronized sensors, fix time offset: -0.015
[ INFO] [1719826597.644682183]: reading paramerter of camera /home/vt008/H2-Mapping/src/dvins/vins_estimator/../config/uav2022/uav_nerf.yaml
[ WARN] [1719826597.644818406]: Waiting for image and imu...
[ WARN] [1719826597.761310770]: Shutdown request received.
[ WARN] [1719826597.761367430]: Reason given for shutdown: [[/rvizvisualisation] Reason: new node registered with same name]
[ WARN] [1719826597.883479979]: First_flag
vt008@vt008:~/H2-Mapping$ [rvizvisualisation-3] process has finished cleanly
log file: /home/vt008/.ros/log/2296f416-378c-11ef-b540-1d303349b635/rvizvisualisation-3*.log
all processes on machine have died, roslaunch will exit
shutting down processing monitor...
... shutting down processing monitor complete
done
[ INFO] [1719826599.088863106]: IMU excitation not enouth!
[ INFO] [1719826599.188844031]: IMU excitation not enouth!
[ INFO] [1719826599.288031656]: IMU excitation not enouth!
[ INFO] [1719826599.386265137]: IMU excitation not enouth!
[ INFO] [1719826599.487149598]: IMU excitation not enouth!
[ INFO] [1719826599.587953864]: IMU excitation not enouth!
[ INFO] [1719826599.685284805]: IMU excitation not enouth!
[ INFO] [1719826599.784328930]: IMU excitation not enouth!
[ INFO] [1719826599.886503276]: IMU excitation not enouth!
[ INFO] [1719826599.993498191]: IMU excitation not enouth!
[ INFO] [1719826600.089161195]: IMU excitation not enough!
..
...
[ WARN] [1719826650.658277016]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658314401]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658327298]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658338554]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658349873]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658361027]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658371225]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658380393]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658389851]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.658417995]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826650.658427813]: Not enough features or parallax; Move device around
[ WARN] [1719826650.758799351]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758842801]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758862575]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758879869]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758896634]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758913277]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758929450]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758945879]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758961752]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.758977714]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826650.758998912]: Not enough features or parallax; Move device around
[ WARN] [1719826650.926113776]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926159911]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926179261]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926197166]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926213521]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926229569]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926248683]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926264815]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926298623]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826650.926315728]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826650.926328264]: Not enough features or parallax; Move device around
[ WARN] [1719826651.023308548]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023371390]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023394127]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023413533]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023433599]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023458189]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023479192]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023500827]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023522551]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.023543234]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826651.023558310]: Not enough features or parallax; Move device around
[ WARN] [1719826651.122757501]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122784723]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122793350]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122800547]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122807282]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122813866]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122821081]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122827942]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122834552]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.122841431]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826651.122846420]: Not enough features or parallax; Move device around
[ WARN] [1719826651.224711872]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224758002]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224789754]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224804902]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224819302]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224834311]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224849902]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224865187]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224913081]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.224931423]: [solveTwoViewGeometry] There is not enough correspondences!
[ INFO] [1719826651.224956884]: Not enough features or parallax; Move device around
...
..
[solveRelativePose] inlier ratio: 1.000000
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
[ WARN] [1719826651.797795352]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797844706]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797861475]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797877716]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797896161]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797916186]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797938164]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797954891]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797971885]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.797998447]: [solveTwoViewGeometry] average parallex: 18.964755
[ INFO] [1719826651.798011831]: Not enough features or parallax; Move device around
[ WARN] [1719826651.890768247]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890809032]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890824454]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890838589]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890856222]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890871224]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890886680]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826651.890901772]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.761905
...
...
[solveRelativePose] inlier ratio: 0.971429
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
[ WARN] [1719826656.928824699]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826656.928892713]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826656.928940065]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826656.928988014]: [solveTwoViewGeometry] There is not enough correspondences!
[ WARN] [1719826656.929021662]: [solveTwoViewGeometry] There is not enough correspondences!
[solveRelativePose] inlier ratio: 0.971429
unstable features tracking, please slowly move you device!
unstable features tracking, please slowly move you device!
[GlobalSFM::evaluateError] ATE P: 0.001736
[GlobalSFM::evaluateError] ATE D: 0.027621
[GlobalSFM::evaluateError] ATE P: 0.001461
[GlobalSFM::evaluateError] ATE D: 0.012003
Ceres Solver Report: Iterations: 8, Initial cost: 4.575857e+02, Final cost: 2.415200e+02, Termination: CONVERGENCE
[ WARN] [1719826656.939701848]: gyroscope bias initial calibration -0.0102977  -0.020526  0.0191045
[ INFO] [1719826656.940736676]:  result g     9.76009 -0.887042  -9.37926  -2.54988
[ INFO] [1719826656.940815591]:  refine     9.805 -0.891124  -9.42242  -2.56161
R0 from g: 7.98443e-16      5.2145    -105.209
[ INFO] [1719826656.942145757]: g0     -1.11022e-16  8.88178e-16        9.805
[ INFO] [1719826656.942158722]: my R0  -1.5947e-15     4.26672    -100.282
[ INFO] [1719826656.955525438]: Initialization finish!
terminate called after throwing an instance of 'cv::Exception'
  what():  OpenCV(4.2.0) ../modules/video/src/lkpyramid.cpp:1257: error: (-215:Assertion failed) (npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function 'calc'

[vins_estimator-1] process has died [pid 13082, exit code -6, cmd /home/vt008/H2-Mapping/devel/lib/vins_estimator/vins_estimator __name:=vins_estimator __log:=/home/vt008/.ros/log/2296f416-378c-11ef-b540-1d303349b635/vins_estimator-1.log].
log file: /home/vt008/.ros/log/2296f416-378c-11ef-b540-1d303349b635/vins_estimator-1*.log

What to do about it?

You asked for a screenshot can you please be more specific so I can post all the screenshots that u need?

ShreyasKuntnal commented 3 months ago

This is the screenshot of visualisation. 1.For live input

python vis/vis_mesh.py \
> $OUTPUT_FOLDER/bak/config.yaml \
> --result_file $OUTPUT_FOLDER \
> -create_mesh

In case of tower.bag file

when I run python script, it ends with cuda memory shortage on my RTX 3060 6GB Graphics card:

In case if I trigger end for the python and let it create a mapping and save it and run visualisation this what it looks like:

According to visualization example gif there should be something about the roam right?

So can you please help me with solving these issues?

Can you please (I request you to) give answers to these issues in the numbered order?

JIANG-CX commented 3 months ago

The mapping is based on the tracking module, so if the tracking is dead, the mapping cannot run. The warning "There is not enough correspondences!" indicates the image is textureless. Let's find the bug step by step. First, could you please use the tower.bag and only run the tracking module (do not run the mapping) and provide the screen shot of rviz?

ShreyasKuntnal commented 3 months ago

First I did rosnode kill -a. Closed roscore terminal. I opened a fresh terminal. and started roscore.

in new terminal tab,

cd H2-Mapping/
source devel/setup.bash
bash ros_cmd/run_vins_rgbd.sh

Note: You can observe there a newline has started but still process is going on in terminal. This behaviour of starting new terminal is new behaviour I don't know whats causing that.

Next in new terminal tab I have started rosbag play tower_compress.orig.bag

What to do after this?

JIANG-CX commented 3 months ago

Ok, it seems the tracking module is installed correctly. So if you use your own realsense data stream, you should also see this rviz visualization, otherwise, the tracking module is not running correctly, you need to check whether you modify the config correctly based on the readme or whether you put your camera in front of a textureless area.

Then you can run the mapping module. If your CUDA memory is limited, you can stop the mapping in advance (ctrl c in the console), then you can check the reconstructed mesh in the mapping/logs.

ShreyasKuntnal commented 3 months ago

so now for the same rosbag i will try running mapping and visualise the same.

For that I have did rosnode kill -a again and restarted roscore, Then again started vins and mapping then started rosbag and let it run for 100s and stopped it.

Then visualized the same this is what it looks like:

So is the visualisation correct?

JIANG-CX commented 3 months ago

Do you find a .ply file in "mapping/logs" ? If so, please provide the screen shot for this .ply file.

ShreyasKuntnal commented 3 months ago

yes

Is that the file where visualisation is stored? How to open that in Linux Ubuntu 20?

ShreyasKuntnal commented 3 months ago

I have updated the above comment with a screenshot is that the right screenshot?

JIANG-CX commented 3 months ago

You can use the meshlab(https://www.meshlab.net/#download) to open it.

ShreyasKuntnal commented 3 months ago

Oh my god it opened. So this is how it looks like. One thing I observed is for every time u start mapping or something we need a fresh start by rosnode kill -a or else it will lead to these errors.

Qs 1. Is this the highest quality possible? Or else any other things can be tuned?

Qs2. Before We close this issue can please help finally with realsense camera live input to please??

JIANG-CX commented 3 months ago

If you do not use 'rosnode kill -a', some programs may not be terminated, which could cause problems.

Answers to your questions: Q1: You can try increasing the "mesh_res" parameter in realsense.yaml. This parameter represents the resolution of the grid used in marching cubes (0.1m/mesh_res). However, the results may not change significantly. Alternatively, you can use a better GPU and increase the "num_iteration" parameter in realsense.yaml. This parameter determines the length of the mapping time.

Q2: The live output from the realsense camera is exactly the same as using the bag. You must first confirm that the tracking is successful (inspect the similar rviz visualization), then run the mapping module and use "ctrl c" to stop it.

ShreyasKuntnal commented 3 months ago

So I recorded this using camera of my table. this is how it looks,

So if I increase mesh res and iteration I would be able to record even more better? Can you please tell me more on what basis can I increase the mesh res and iterations? As you said 0.1m/mesh_res what does that mean and what are other things that I need to increase to get a good quality output?

JIANG-CX commented 3 months ago

The quality of this work largely depends on the quality of the depth data and the accuracy of the pose estimation. Based on the results you provided, the depth data may not be great. You might try using more accurate RGB-D sensors like the L515.

You can find the details of the usage of mesh_res and num_iteration in the code. Briefly speaking, mesh_res is used to determine the grid resolution when using the marching cubes algorithm to generate the mesh (you can Google "marching cubes" for more details). num_iteration controls the number of optimization iterations per frame. Since it is a datastream input, a longer training time means more frames might be neglected as new frames arrive after completing the num_iteration iterations. Therefore, if you have a more powerful GPU, you can increase the num_iteration value, as it will run faster and fewer regions will be neglected.

ShreyasKuntnal commented 3 months ago

Is setting up this repo on jetson the same? Oh ok Thank you so much for your faster.

JIANG-CX commented 3 months ago

The same. The provided bag file was recorded during our Jetson experiment.

ShreyasKuntnal commented 3 months ago

Oh ok thank you so much sir. Even the tiny cudnn and everyother thing will be the same right?

JIANG-CX commented 3 months ago

The same. The thing you need to notice is that you should install pytorch in https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048

SYSU-STAR / H2-Mapping

Needed help to start H2-Mapping for realsense camera live data. Getting some error #38