search

SYSU-STAR / H2-Mapping

H2-Mapping: Real-time Dense Mapping Using Hierarchical Hybrid Representation (2023 RAL Best Paper Award)
GNU General Public License v3.0
362 stars 12 forks source link

自建数据集可视化过程报错 #13

Closed lvmingzhe closed 9 months ago

lvmingzhe commented 9 months ago
  1. 准备运行采集程序和Visualization之前已在 /home/hello/code/H2-Mapping/mapping/configs/l515/realsense.yaml设置save_ckpt_freq=10
  2. 已修改svo文件 
    torch.classes.load_library("/home/hello/code/H2-Mapping/mapping/third_party/sparse_octree/build/lib.linux-aarch64-cpython-38/svo.cpython-38-aarch64-linux-gnu.so")
  3. replica样例可视化正常.
  4. 自建数据集可视化命令： 
    python vis/vis_mesh.py /home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V/bak/config.yaml --result_file /home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V -create_mesh
  5. 报错 
    (h2mapping) hello@hello-desktop:~/code/H2-Mapping/mapping$ python vis/vis_mesh.py /home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V/bak/config.yaml --result_file /home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V -create_mesh
realsense
Traceback (most recent call last):
File "vis/vis_mesh.py", line 44, in <module>
data_stream = get_dataset(args)
File "/home/hello/code/H2-Mapping/mapping/src/utils/import_util.py", line 7, in get_dataset
Dataset = import_module("dataset." + args.dataset)
File "/home/hello/mambaforge/envs/h2mapping/lib/python3.8/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
File "<frozen importlib._bootstrap>", line 991, in _find_and_load
File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'dataset.realsense'
lvmingzhe commented 9 months ago

相关配置文件

  1. /home/hello/code/H2-Mapping/mapping/configs/l515/tower.yaml 
    
base_config: configs/l515/realsense.yaml

exp_name: realsense/l515

data_specs: data_path: '../Datasets/realsense/tower' max_depth: 3 use_gt: True # notice

decoder_specs: bound: [ [ 0.0,20.0 ],[ 0.0,20.0 ],[ 0.0,10.0 ] ] # [x_min,x_max],[y_min,y_max],[z_min,z_max]


2. ```/home/hello/code/H2-Mapping/mapping/configs/l515/realsense.yaml```

log_dir: './logs' decoder: parallel_hash_net dataset: l515 update_pose: False

criteria: rgb_weight: 0.5 # .5 depth_weight: 1 sdf_weight: 50000.0 fs_weight: 10.0 sdf_truncation: 0.05 # 0.1

decoder_specs: voxel_size: 0.1 # same as mapper_specs L: 4 # Number of levels F_entry: 2 # Number of feature dimensions per entry log2_T: 19 # each level's hashmap_size = F_entry * (2*F_entry) b: 2.0 # each level's resolution = N_min (b**Level)

mapper_specs: start_frame: 0 end_frame: -1 N_rays_each: 4096 # mapping's sampling ray batch_size: 10240 num_vertexes: 200000 inflate_margin_ratio: 0.1 voxel_size: 0.1 step_size: 0.1 num_iterations: 8 max_voxel_hit: 10 final_iter: 0 mesh_res: 8 overlap_th: 0.8 kf_window_size: 8 kf_selection_method: "multiple_max_set_coverage" # "random” or “multiple_max_set_coverage” kf_selection_random_radio: 0.5 # random keyframe ratio insert_method: "intersection" # "naive" or "intersection" insert_ratio: 0.85 offset: 10 # used to make make the coordinate of each point positive use_adaptive_ending: True # adaptive iteration

ros_args: intrinsic: [ 605.688720703125, 606.214965820312, 324.855010986328, 236.482986450195 ] # K[0, 0], K[1, 1], K[0, 2], K[1, 2] color_topic: '/camera/color/image_raw' depth_topic: '/camera/aligned_depth_to_color/image_raw' pose_topic: '/vins_estimator/cam_pose'

debug_args: verbose: false mesh_freq: -1 render_freq: -1 save_ckpt_freq: 10 render_res: [ 320, 240 ]

lvmingzhe commented 9 months ago

/home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V 文件夹下有四个子文件夹

bak  ckpt  imgs  mesh  misc

其中ckpt

(base) hello@hello-desktop:~/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-17-10-29-44V/ckpt$ ls
000009.pth  000069.pth  000129.pth  000189.pth  000249.pth  000309.pth
000019.pth  000079.pth  000139.pth  000199.pth  000259.pth  000319.pth
000029.pth  000089.pth  000149.pth  000209.pth  000269.pth  000329.pth
000039.pth  000099.pth  000159.pth  000219.pth  000279.pth  final_ckpt.pth
000049.pth  000109.pth  000169.pth  000229.pth  000289.pth
000059.pth  000119.pth  000179.pth  000239.pth  000299.pth

bak文件夹内的config.yaml文件

base_config: configs/l515/realsense.yaml
config: configs/l515/tower.yaml
criteria:
  depth_weight: 1
  fs_weight: 10.0
  rgb_weight: 0.5
  sdf_truncation: 0.05
  sdf_weight: 50000.0
data_specs:
  data_path: ../Datasets/realsense/tower
  max_depth: 3
  use_gt: true
dataset: realsense
debug: false
debug_args:
  mesh_freq: -1
  render_freq: -1
  render_res:
  - 320
  - 240
  save_ckpt_freq: 10
  verbose: false
decoder: parallel_hash_net
decoder_specs:
  F_entry: 2
  L: 4
  b: 2.0
  bound:
  - - 0.0
    - 20.0
  - - 0.0
    - 20.0
  - - 0.0
    - 10.0
  log2_T: 19
  voxel_size: 0.1
exp_name: realsense/l515
log_dir: ./logs
mapper_specs:
  N_rays_each: 4096
  batch_size: 10240
  end_frame: -1
  final_iter: 0
  inflate_margin_ratio: 0.1
  insert_method: intersection
  insert_ratio: 0.85
  kf_selection_method: multiple_max_set_coverage
  kf_selection_random_radio: 0.5
  kf_window_size: 8
  max_voxel_hit: 10
  mesh_res: 8
  num_iterations: 8
  num_vertexes: 200000
  offset: 10
  overlap_th: 0.8
  start_frame: 0
  step_size: 0.1
  use_adaptive_ending: true
  voxel_size: 0.1
resume: null
ros_args:
  color_topic: /camera/color/image_raw
  depth_topic: /camera/aligned_depth_to_color/image_raw
  intrinsic:
  - 605.688720703125
  - 606.214965820312
  - 324.855010986328
  - 236.482986450195
  pose_topic: /vins_estimator/cam_pose
run_ros: true
update_pose: false
JIANG-CX commented 9 months ago

Bug已修复，谢谢

lvmingzhe commented 9 months ago

replica可以正常显示

e3f0dd2cf5c3ed98bfff8a6f1acbcf4

自建数据现在可以运行不报错了，但是只显示了轨迹，没有背景mesh

2509fb2b21b490c8e6fc3d2392a6bb9

我检查了中间的mesh，似乎是正常的. image

目前的bak文件

base_config: configs/l515/realsense.yaml
config: configs/l515/tower.yaml
criteria:
  depth_weight: 1
  fs_weight: 10.0
  rgb_weight: 0.5
  sdf_truncation: 0.05
  sdf_weight: 50000.0
data_specs:
  max_depth: 3
  use_gt: true
dataset: l515
debug: false
debug_args:
  mesh_freq: -1
  render_freq: -1
  render_res:
  - 320
  - 240
  save_ckpt_freq: 10
  verbose: false
decoder: parallel_hash_net
decoder_specs:
  F_entry: 2
  L: 4
  b: 2.0
  bound:
  - - 0.0
    - 20.0
  - - 0.0
    - 20.0
  - - 0.0
    - 10.0
  log2_T: 19
  voxel_size: 0.1
exp_name: realsense/l515
log_dir: ./logs
mapper_specs:
  N_rays_each: 4096
  batch_size: 10240
  end_frame: -1
  final_iter: 0
  inflate_margin_ratio: 0.1
  insert_method: intersection
  insert_ratio: 0.85
  kf_selection_method: multiple_max_set_coverage
  kf_selection_random_radio: 0.5
  kf_window_size: 8
  max_voxel_hit: 10
  mesh_res: 8
  num_iterations: 8
  num_vertexes: 200000
  offset: 10
  overlap_th: 0.8
  start_frame: 0
  step_size: 0.1
  use_adaptive_ending: true
  voxel_size: 0.1
resume: null
ros_args:
  color_topic: /camera/color/image_raw
  depth_topic: /camera/aligned_depth_to_color/image_raw
  intrinsic:
  - 605.688720703125
  - 606.214965820312
  - 324.855010986328
  - 236.482986450195
  pose_topic: /vins_estimator/cam_pose
run_ros: true
update_pose: false

这个/home/hello/code/H2-Mapping/mapping/logs/realsense/l515/2023-10-18-11-47-37序列没有重新运行，是原来保存的数据包 请帮忙分析一下是不是还有什么设置没对的问题？谢谢

JIANG-CX commented 9 months ago

有可能是因为mesh比较大，可视化会比较慢，可以试一下把在vis_mesh.py的91行（https://github.com/SYSU-STAR/H2-Mapping/blob/main/mapping/vis/vis_mesh.py#L91）的下面加一个time.sleep函数来延时

lvmingzhe commented 9 months ago

经过尝试，调大88行的time.sleep()延时有帮助~ 另外，想请教一下figure里边的room0_mesh_out.gif是如何生成的？ 是用软件录屏然后转gif吗

JIANG-CX commented 9 months ago

经过尝试，调大88行的time.sleep()延时有帮助~ 另外，想请教一下figure里边的room0_mesh_out.gif是如何生成的？ 是用软件录屏然后转gif吗

可以录屏，也可以通过open3d直接获得窗口显示的图片，代码中留了一个小接口，也就是save_rendering这个参数，可以保存窗口显示的图片。最后都是图片或者视频转gif。

lvmingzhe commented 9 months ago

好的，--save_rendering是可以直接输出jpg图片。但是图片数量过多转gif的话就很容易系统卡死，应该还是转成视频好一点。

JIANG-CX commented 9 months ago

嗯嗯

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: Mingzhe LV @.> 发送时间: Wednesday, October 18, 2023 11:47:03 PM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 自建数据集可视化过程报错 (Issue #13)

好的，--save_rendering是可以直接输出jpg图片。但是图片数量过多转gif的话就很容易系统卡死，应该还是转成视频好一点。

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/13#issuecomment-1768770212, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNXCJCHXZ2RXIJMK2CDX772XPAVCNFSM6AAAAAA6EZCAFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONRYG43TAMRRGI. You are receiving this because you commented.Message ID: @.***>

jiahao-hu commented 7 months ago

debug: false debug_args: mesh_freq: -1 render_freq: -1 render_res:

  • 320
  • 240

您好请问如何自制replica格式的数据集

JIANG-CX commented 7 months ago

debug: false debug_args: mesh_freq: -1 render_freq: -1 render_res:

  • 320
  • 240

您好请问如何自制replica格式的数据集 请参考https://github.com/SYSU-STAR/H2-Mapping/issues/19

jiahao-hu commented 7 months ago

debug: false debug_args: mesh_freq: -1 render_freq: -1 render_res:

  • 320
  • 240

您好请问如何自制replica格式的数据集 请参考#19

您好我还想问一个问题，我在运行tower_compress.bag的tracking module时显示

[ WARN] [1702976699.548338534]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548386447]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548397526]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548424609]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548439116]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548453115]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548470016]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548486254]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548515426]: [solveTwoViewGeometry] average parallex: 13.941514 [ WARN] [1702976699.548531513]: [solveTwoViewGeometry] average parallex: 13.164767 [ INFO] [1702976699.548540498]: Not enough features or parallax; Move device around [ WARN] [1702976701.040162580]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040202165]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040211876]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040219152]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040226128]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 1.000000 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [ WARN] [1702976701.049264840]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049284105]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049291423]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049297752]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 0.978723 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [ WARN] [1702976701.674828076]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.674873721]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 1.000000 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [GlobalSFM::evaluateError] ATE P: 0.000550 [GlobalSFM::evaluateError] ATE D: 0.012062 [GlobalSFM::evaluateError] ATE P: 0.000515 [GlobalSFM::evaluateError] ATE D: 0.005614 Ceres Solver Report: Iterations: 6, Initial cost: 3.550318e+02, Final cost: 1.824717e+02, Termination: CONVERGENCE [ WARN] [1702976701.698973881]: gyroscope bias initial calibration 0.029841 -0.0123248 0.00800037 [ INFO] [1702976701.701680997]: result g 9.59421 0.343591 -9.35466 -2.10267 [ INFO] [1702976701.701764973]: refine 9.805 0.35114 -9.56018 -2.14886 R0 from g: 0 -2.05233 -102.668 [ INFO] [1702976701.704682781]: g0 5.89806e-17 8.88178e-16 9.805 [ INFO] [1702976701.704694358]: my R0 1.74038e-16 1.96061 -103.558 [ INFO] [1702976701.733253201]: Initialization finish! [ WARN] [1702976723.096605885]: image discontinue! reset the feature tracker! [FATAL] [1702976723.096642519]: ASSERTION FAILED file = /opt/ros/noetic/include/ros/publisher.h line = 107 cond = false message = [FATAL] [1702976723.096650577]: Call to publish() on an invalid Publisher [FATAL] [1702976723.096656689]: 这个问题怎么解决呢

JIANG-CX commented 7 months ago

debug: false debug_args: mesh_freq: -1 render_freq: -1 render_res:

  • 320
  • 240

您好请问如何自制replica格式的数据集 请参考#19

您好我还想问一个问题，我在运行tower_compress.bag的tracking module时显示 [ WARN] [1702976696.185587985]: First_flag [ WARN] [1702976699.548338534]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548386447]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548397526]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548424609]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548439116]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548453115]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548470016]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548486254]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976699.548515426]: [solveTwoViewGeometry] average parallex: 13.941514 [ WARN] [1702976699.548531513]: [solveTwoViewGeometry] average parallex: 13.164767 [ INFO] [1702976699.548540498]: Not enough features or parallax; Move device around [ WARN] [1702976701.040162580]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040202165]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040211876]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040219152]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.040226128]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 1.000000 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [ WARN] [1702976701.049264840]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049284105]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049291423]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.049297752]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 0.978723 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [ WARN] [1702976701.674828076]: [solveTwoViewGeometry] There is not enough correspondences! [ WARN] [1702976701.674873721]: [solveTwoViewGeometry] There is not enough correspondences! [solveRelativePose] inlier ratio: 1.000000 unstable features tracking, please slowly move you device! unstable features tracking, please slowly move you device! [GlobalSFM::evaluateError] ATE P: 0.000550 [GlobalSFM::evaluateError] ATE D: 0.012062 [GlobalSFM::evaluateError] ATE P: 0.000515 [GlobalSFM::evaluateError] ATE D: 0.005614 Ceres Solver Report: Iterations: 6, Initial cost: 3.550318e+02, Final cost: 1.824717e+02, Termination: CONVERGENCE [ WARN] [1702976701.698973881]: gyroscope bias initial calibration 0.029841 -0.0123248 0.00800037 [ INFO] [1702976701.701680997]: result g 9.59421 0.343591 -9.35466 -2.10267 [ INFO] [1702976701.701764973]: refine 9.805 0.35114 -9.56018 -2.14886 R0 from g: 0 -2.05233 -102.668 [ INFO] [1702976701.704682781]: g0 5.89806e-17 8.88178e-16 9.805 [ INFO] [1702976701.704694358]: my R0 1.74038e-16 1.96061 -103.558 [ INFO] [1702976701.733253201]: Initialization finish! [ WARN] [1702976723.096605885]: image discontinue! reset the feature tracker! [FATAL] [1702976723.096642519]: ASSERTION FAILED file = /opt/ros/noetic/include/ros/publisher.h line = 107 cond = false message = [FATAL] [1702976723.096650577]: Call to publish() on an invalid Publisher [FATAL] [1702976723.096656689]: 这个问题怎么解决呢

请参考https://github.com/SYSU-STAR/H2-Mapping/issues/14