rpng / open_vins

An open source platform for visual-inertial navigation research.
https://docs.openvins.com
GNU General Public License v3.0
2.22k stars 652 forks source link

Issue with roslaunch ov_msckf subscriber.launch Stalling #469

Closed mo-aqeel closed 2 months ago

mo-aqeel commented 2 months ago

Description: I am facing an issue when I run the _roslaunch ovmsckf subscriber.launch file with my custom configuration path. The process freezes at the stage shown in the terminal output below. Camera and IMU topics are being published successfully, despite some warnings when launching the stereo camera configuration.

_FLIR Camera Model: BFLY-U3-23S6C-C (stereo configuration)
_ROS Version: Noetic_
_IMU: Xsens MTi 600 series__

Since this is my first time configuring camera with ROS running OpenVINS, I'm unsure if the issue is with my configuration or a driver limitation.

launch

Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://aqubuITE-AX:35875/

SUMMARY

CLEAR PARAMETERS

PARAMETERS

NODES / ov_msckf (ov_msckf/run_subscribe_msckf)

auto-starting new master process[master]: started with pid [60920] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 7167949a-71cc-11ef-bb2b-0f226cf80f78 process[rosout-1]: started with pid [60931] started core service [/rosout] process[ov_msckf-2]: started with pid [60939] overriding node verbosity with value from ROS! Setting printing level to: INFO overriding node max_cameras with value from ROS! overriding node max_cameras with value from ROS! overriding node use_stereo with value from ROS! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! overriding node record_timing_information with value from ROS! overriding node record_timing_filepath with value from ROS! overriding node use_stereo with value from ROS! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! subscribing to IMU: /imu/data subscribing to cam (stereo): /stereo/right/image_raw subscribing to cam (stereo): /stereo/left/image_raw

after ctrl+c

^C[ov_msckf-2] killing on exit camera-imu timeoffset = -0.02110

cam0 intrinsics: 1180.291,1189.487,897.779,610.113 -0.32058,0.11265,-0.00370,0.01269

cam1 intrinsics: 1187.671,1196.000,1001.743,620.130 -0.34569,0.13851,-0.00375,-0.00654

T_C0toI: -0.047,-0.010,-0.999,-0.045, 0.999,-0.008,-0.047,0.066, -0.008,-1.000,0.010,0.000, 0.000,0.000,0.000,1.000

T_C1toI: 0.023,0.013,-1.000,-0.044, 1.000,-0.004,0.023,-0.081, -0.004,-1.000,-0.014,-0.000, 0.000,0.000,0.000,1.000

TIME: 9223372036854.775 seconds

terminate called after throwing an instance of 'boost::wrapexcept' what(): boost: mutex lock failed in pthread_mutex_lock: Invalid argument [rosout-1] killing on exit ^C[master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done aqubu

aidadgy commented 1 month ago

Description: I am facing an issue when I run the _roslaunch ovmsckf subscriber.launch file with my custom configuration path. The process freezes at the stage shown in the terminal output below. Camera and IMU topics are being published successfully, despite some warnings when launching the stereo camera configuration.

_FLIR Camera Model: BFLY-U3-23S6C-C (stereo configuration)
_ROS Version: Noetic_
_IMU: Xsens MTi 600 series__

Since this is my first time configuring camera with ROS running OpenVINS, I'm unsure if the issue is with my configuration or a driver limitation.

launch

<!-- what config we are going to run (should match folder name) -->
<arg name="verbosity"   default="INFO" /> <!-- ALL, DEBUG, INFO, WARNING, ERROR, SILENT -->
<arg name="config"      default="euroc_mav" /> <!-- euroc_mav, tum_vi, rpng_aruco -->
<arg name="config_path" default="$(find ov_msckf)/../config/$(arg config)/estimator_config.yaml" />

<!-- mono or stereo and what ros bag to play -->
<arg name="max_cameras" default="2" />
<arg name="use_stereo"  default="true" />
<arg name="bag_start"   default="0" /> <!-- v1-2: 0, mh1: 40, mh2: 35, mh3: 17.5, mh4-5: 15 -->
<arg name="bag_rate"    default="1" />
<arg name="dataset"     default="V1_01_easy" /> <!-- V1_01_easy, V1_02_medium, V2_02_medium -->
<arg name="dobag"       default="false" /> <!-- if we should play back the bag -->
<arg name="bag"         default="$(env HOME)/Downloads/$(arg dataset).bag" />
<!-- saving trajectory path and timing information -->
<arg name="dosave"      default="false" />
<arg name="dotime"      default="false" />
<arg name="path_est"    default="/tmp/traj_estimate.txt" />
<arg name="path_time"   default="/tmp/traj_timing.txt" />

<!-- if we should viz the groundtruth -->
<arg name="dolivetraj"  default="false" />
<arg name="path_gt"     default="$(find ov_data)/$(arg config)/$(arg dataset).txt" />

<!-- MASTER NODE! -->
<node name="ov_msckf" pkg="ov_msckf" type="run_subscribe_msckf" output="screen" clear_params="true" required="true">

    <!-- master configuration object -->
    <param name="verbosity"              type="string" value="$(arg verbosity)" />
    <param name="config_path"            type="string" value="$(arg config_path)" />

    <!-- world/filter parameters -->
    <param name="use_stereo"             type="bool"   value="$(arg use_stereo)" />
    <param name="max_cameras"            type="int"    value="$(arg max_cameras)" />

    <!-- timing statistics recording -->
    <param name="record_timing_information"   type="bool"   value="$(arg dotime)" />
    <param name="record_timing_filepath"      type="string" value="$(arg path_time)" />

</node>

<!-- play the dataset -->
<!-- <group if="$(arg dobag)">
    <node pkg="rosbag" type="play" name="rosbag" args="-d 1 -r $(arg bag_rate) -s $(arg bag_start) $(arg bag)" required="true"/>
</group> -->

<!-- record the trajectory if enabled -->
<!-- <group if="$(arg dosave)">
    <node name="recorder_estimate" pkg="ov_eval" type="pose_to_file" output="screen" required="true">
        <param name="topic"      type="str" value="/ov_msckf/poseimu" />
        <param name="topic_type" type="str" value="PoseWithCovarianceStamped" />
        <param name="output"     type="str" value="$(arg path_est)" />
    </node>
</group> -->

<!-- path viz of aligned gt -->
<!-- <group if="$(arg dolivetraj)">
    <node name="live_align_trajectory" pkg="ov_eval" type="live_align_trajectory" output="log" clear_params="true">
        <param name="alignment_type" type="str" value="posyaw" />
        <param name="path_gt"        type="str" value="$(arg path_gt)" />
    </node>
</group> -->

Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://aqubuITE-AX:35875/

SUMMARY

CLEAR PARAMETERS

  • /ov_msckf/

PARAMETERS

  • /ov_msckf/config_path: /home/aqubu/works...
  • /ov_msckf/max_cameras: 2
  • /ov_msckf/record_timing_filepath: /tmp/traj_timing.txt
  • /ov_msckf/record_timing_information: False
  • /ov_msckf/use_stereo: True
  • /ov_msckf/verbosity: INFO
  • /rosdistro: noetic
  • /rosversion: 1.16.0

NODES / ov_msckf (ov_msckf/run_subscribe_msckf)

auto-starting new master process[master]: started with pid [60920] ROS_MASTER_URI=http://localhost:11311

setting /run_id to 7167949a-71cc-11ef-bb2b-0f226cf80f78 process[rosout-1]: started with pid [60931] started core service [/rosout] process[ov_msckf-2]: started with pid [60939] overriding node verbosity with value from ROS! Setting printing level to: INFO overriding node max_cameras with value from ROS! overriding node max_cameras with value from ROS! overriding node use_stereo with value from ROS! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! overriding node record_timing_information with value from ROS! overriding node record_timing_filepath with value from ROS! overriding node use_stereo with value from ROS! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! parameter T_imu_cam not found, trying T_cam_imu instead (will return T_imu_cam still)! subscribing to IMU: /imu/data subscribing to cam (stereo): /stereo/right/image_raw subscribing to cam (stereo): /stereo/left/image_raw

after ctrl+c

^C[ov_msckf-2] killing on exit camera-imu timeoffset = -0.02110

cam0 intrinsics: 1180.291,1189.487,897.779,610.113 -0.32058,0.11265,-0.00370,0.01269

cam1 intrinsics: 1187.671,1196.000,1001.743,620.130 -0.34569,0.13851,-0.00375,-0.00654

T_C0toI: -0.047,-0.010,-0.999,-0.045, 0.999,-0.008,-0.047,0.066, -0.008,-1.000,0.010,0.000, 0.000,0.000,0.000,1.000

T_C1toI: 0.023,0.013,-1.000,-0.044, 1.000,-0.004,0.023,-0.081, -0.004,-1.000,-0.014,-0.000, 0.000,0.000,0.000,1.000

TIME: 9223372036854.775 seconds

terminate called after throwing an instance of 'boost::wrapexceptboost::lock_error' what(): boost: mutex lock failed in pthread_mutex_lock: Invalid argument [rosout-1] killing on exit ^C[master] killing on exit shutting down processing monitor... ... shutting down processing monitor complete done aqubu

Hello author, I am running an open source code, there is also a similar error, when the program is running after:/usr/include/boost/thread/pthread/recursive_mutex.hpp:108: void boost::recursive_mutex::lock(): Assertion `!posix::pthread_mutex_lock(&m)' failed., how do you solve your error, the umbilical cord can receive your reply

mo-aqeel commented 1 month ago

The issue was that the timestamps of the cameras and the IMU were not aligned, as the IMU used a relative timestamp. To resolve this, I converted the IMU timestamp to epoch time to match the camera’s timestamp. Hope this helps

aidadgy commented 1 month ago

Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is

Thank you very much for your answer. I also found some problems on the camera and IMU timestamps before. Meanwhile, I would like to further know where in the source code you made some modifications, which will help to solve the errors quickly, thank you very much

mo-aqeel commented 1 month ago

I believe you have to make changes in the void ROS1Visualizer::callback_inertial(const sensor_msgs::Imu::ConstPtr &msg) in the ROS1Visualizer.cpp file. You just have to change the IMU timestamp to match the camera timestamp.

On Thu, Oct 24, 2024 at 14:29 aidadgy @.***> wrote:

Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is

Thank you very much for your answer. I also found some problems on the camera and IMU timestamps before. Meanwhile, I would like to further know where in the source code you made some modifications, which will help to solve the errors quickly, thank you very much

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/469#issuecomment-2434334110, or unsubscribe https://github.com/notifications/unsubscribe-auth/BLIRTHVJ5O4JMCSZSEVVDXLZ5CAUVAVCNFSM6AAAAABOFN34WKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGMZTIMJRGA . You are receiving this because you modified the open/close state.Message ID: @.***>

aidadgy commented 1 month ago

I believe you have to make changes in the void ROS1Visualizer::callback_inertial(const sensor_msgs::Imu::ConstPtr &msg) in the ROS1Visualizer.cpp file. You just have to change the IMU timestamp to match the camera timestamp. On Thu, Oct 24, 2024 at 14:29 aidadgy @.> wrote: Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is Thank you very much for your answer. I also found some problems on the camera and IMU timestamps before. Meanwhile, I would like to further know where in the source code you made some modifications, which will help to solve the errors quickly, thank you very much — Reply to this email directly, view it on GitHub <#469 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BLIRTHVJ5O4JMCSZSEVVDXLZ5CAUVAVCNFSM6AAAAABOFN34WKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGMZTIMJRGA . You are receiving this because you modified the open/close state.Message ID: @.>

Thank you for your answer. Is the data you use public data or your own data? If it is a public data set, theoretically the time stamps should be aligned, and there should be no mismatch between the imu and the camera's time stamps

mo-aqeel commented 4 weeks ago

Hello, I was using my own data not public data.

On Mon, Oct 28, 2024 at 20:23 aidadgy @.***> wrote:

I believe you have to make changes in the void ROS1Visualizer::callback_inertial(const sensormsgs::Imu::ConstPtr &msg) in the ROS1Visualizer.cpp file. You just have to change the IMU timestamp to match the camera timestamp. … <#m-5927503638530960477_> On Thu, Oct 24, 2024 at 14:29 aidadgy @.> wrote: Logging output roslaunch ov_msckf subscribe.launch config:=test_xr ... logging to /home/aqubu/.ros/log/7167949a-71cc-11ef-bb2b-0f226cf80f78/roslaunch-aqubuITE-AX-60911.log Checking log directory for disk usage. This may take a while. Press Ctrl-C to interrupt Done checking log file disk usage. Usage is Thank you very much for your answer. I also found some problems on the camera and IMU timestamps before. Meanwhile, I would like to further know where in the source code you made some modifications, which will help to solve the errors quickly, thank you very much — Reply to this email directly, view it on GitHub <#469 (comment) https://github.com/rpng/open_vins/issues/469#issuecomment-2434334110>, or unsubscribe https://github.com/notifications/unsubscribe-auth/BLIRTHVJ5O4JMCSZSEVVDXLZ5CAUVAVCNFSM6AAAAABOFN34WKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGMZTIMJRGA https://github.com/notifications/unsubscribe-auth/BLIRTHVJ5O4JMCSZSEVVDXLZ5CAUVAVCNFSM6AAAAABOFN34WKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGMZTIMJRGA . You are receiving this because you modified the open/close state.Message ID: @.>

Thank you for your answer. Is the data you use public data or your own data? If it is a public data set, theoretically the time stamps should be aligned, and there should be no mismatch between the imu and the camera's time stamps

— Reply to this email directly, view it on GitHub https://github.com/rpng/open_vins/issues/469#issuecomment-2441311888, or unsubscribe https://github.com/notifications/unsubscribe-auth/BLIRTHWA3VYK327J3V2YH4DZ5YNCZAVCNFSM6AAAAABOFN34WKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBRGMYTCOBYHA . You are receiving this because you modified the open/close state.Message ID: @.***>