zfkdsg commented 1 month ago

当我想运行自己的数据的时候发生了下面的错误 ERROR] [1729741188.250898]: bad callback: <bound method Subscriber.callback of <message_filters.Subscriber object at 0x7fcafad6ad90>> Traceback (most recent call last): File "/opt/ros/noetic/lib/python3/dist-packages/rospy/topics.py", line 750, in _invoke_callback cb(msg) File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/init.py", line 76, in callback self.signalMessage(msg) File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/init.py", line 58, in signalMessage cb((msg + args)) File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/init.py", line 330, in add self.signalMessage(msgs) File "/opt/ros/noetic/lib/python3/dist-packages/message_filters/init.py", line 58, in signalMessage cb(*(msg + args)) File "/home/dlut/zfk/H2-Mapping/mapping/src/mapping.py", line 137, in callback self.mapping_step(self.idx, tracked_frame, update_pose) #之后进入构图环节 File "/home/dlut/zfk/H2-Mapping/mapping/src/mapping.py", line 157, in mapping_step self.do_mapping(tracked_frame=tracked_frame, update_pose=update_pose) #构建地图并对权重新型了 File "/home/dlut/zfk/H2-Mapping/mapping/src/mapping.py", line 286, in do_mapping bundle_adjust_frames( File "/home/dlut/zfk/H2-Mapping/mapping/src/functions/render_helpers.py", line 463, in bundle_adjust_frames assert (hits.sum() > 0) AssertionError 能不能帮我看看是什么问题万分感谢

zfkdsg commented 1 month ago

好像是体素大小设置的太小了导致的

JIANG-CX commented 1 month ago

这个是因为所有的采样射线都没有相应的sdf prior的voxel与之对应。可能是深度图很稀疏同时体素较小

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Thursday, October 24, 2024 2:09:11 PM To: SYSU-STAR/H2-Mapping @.> Cc: Subscribed @.***> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

好像是体素大小设置的太小了导致的

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2434380413, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNVHWF4ARDA6O3OEWPLZ5CFIPAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZUGM4DANBRGM. You are receiving this because you are subscribed to this thread.Message ID: @.***>

zfkdsg commented 1 month ago

作者你好感谢你的回复我今天尝试采用D435i相机驱动H2-mapping 首先我的问题是首先我要修改uav_nerf.yaml里面的内参和imu到相机的旋转平移矩阵我采用了两种方式输出color 的内参得到的结果却不一样第一种是rs-enumerate-devices -c 输出如下： Intrinsic of "Color" / 1920x1080 / {YUYV/RGB8/BGR8/RGBA8/BGRA8/Y8} Width: 1920 Height: 1080 PPX: 965.241149902344 PPY: 565.583923339844 Fx: 1369.13146972656 Fy: 1369.22021484375 Distortion: Inverse Brown Conrady Coeffs: 0 0 0 0 0
FOV (deg): 70.07 x 43.03 这里显示我的相机是 19201080 对应的内参如上但是由于我的ros1消息 color/image_raw 的宽和高是640480 因此经过网上查询应该修改内参经过计算我修改uav_nerf.yaml内参如下：

projection_parameters:

fx: 456.377

fy: 607.001

cx: 321.747

cy: 251.001

以及 realsense 的 ros_args里面的参数 intrinsic: [456.377, 607.001, 321.747, 251.001]

然而我在网上查询还有一种查看内参的方式 ros topic camera/camera/info 也可以查看相机内参得到的结果却是： height: 720 width: 1280 distortion_model: plumb_bob d:

0.0
0.0
0.0
0.0
0.0 k:
912.7542724609375
0.0
643.494140625
0.0
912.8134765625
377.0559387207031
0.0
0.0
1.0

这里我就不知道用哪个了还请指教谢谢

JIANG-CX commented 1 month ago

rs-enumerate-devices -c应该是会显示不同分辨率设置下对应的内参的，每一个分辨率都会有一个内参显示在terminal中，选用realsense所使用的分辨率的内参即可

zfkdsg commented 1 month ago

您好昨天我又重新采了一份数据但是再运行slam的过程中出现如下报错： terminate called after throwing an instance of 'cv::Exception' what(): OpenCV(4.2.0) ../modules/video/src/lkpyramid.cpp:1257: error: (-215:Assertion failed) (npoints = prevPtsMat.checkVector(2, CV_32F, true)) >= 0 in function 'calc' [vins_estimator-1] process has died [pid 2576239, exit code -6, cmd /home/dlut/zfk/H2-Mapping/devel/lib/vins_estimator/vins_estimator __name:=vins_estimator __log:=/home/dlut/.ros/log/e24ce52c-7198-11ef-9ac7-b391ef66e5ba/vins_estimator-1.log]. log file: /home/dlut/.ros/log/e24ce52c-7198-11ef-9ac7-b391ef66e5ba/vins_estimator-1*.log 请问这是什么问题万分感谢

zfkdsg commented 1 month ago

而且代码运行的过程中一直输出 wrong size

JIANG-CX commented 1 month ago

vins的tracking没跑通，看报错是特征点数量的问题，再检查一下内外参

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 10:32:15 AM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

而且代码运行的过程中一直输出 wrong size

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2436688303, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNRZ4SS7DAQHYDZUFL3Z5GUS7AVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZWGY4DQMZQGM. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

我的流程如下：首先确定我的电脑图片是 640*480 因此我通过命令输出如下： Intrinsic of "Color" / 640x480 / {YUYV/RGB8/BGR8/RGBA8/BGRA8/Y8} Width: 640 Height: 480 PPX: 322.329406738281 PPY: 251.370620727539 Fx: 608.502868652344 Fy: 608.542358398438 Distortion: Inverse Brown Conrady Coeffs: 0 0 0 0 0
FOV (deg): 55.48 x 43.03 因此我进行了修改： vins: distortion_parameters: k1: 0 k2: 0 p1: 0 p2: 0 projection_parameters: fx: 608.502868652344 fy: 608.542358398438 cx: 322.329406738281 cy: 251.370620727539 nerf: ros_args:
intrinsic: [608.502868652344,608.542358398438, 322.329406738281, 251.370620727539] # K[0, 0], K[1, 1], K[0, 2], K[1, 2] 同理我修改了imu数据

我录了两个数据包第一个数据包vins的tracking运行没有问题而第二个包在只有墙面的时候报了错误目前我分析是因为没有特征点导致slam运行出错我现在重新录制了一个数据包再进行尝试

JIANG-CX commented 1 month ago

嗯，没有特征点会有问题

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 2:27:02 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

我的流程如下：首先确定我的电脑图片是 640*480 因此我通过命令输出如下： Intrinsic of "Color" / 640x480 / {YUYV/RGB8/BGR8/RGBA8/BGRA8/Y8} Width: 640 Height: 480 PPX: 322.329406738281 PPY: 251.370620727539 Fx: 608.502868652344 Fy: 608.542358398438 Distortion: Inverse Brown Conrady Coeffs: 0 0 0 0 0 FOV (deg): 55.48 x 43.03 因此我进行了修改： vins: distortion_parameters: k1: 0 k2: 0 p1: 0 p2: 0 projection_parameters: fx: 608.502868652344 fy: 608.542358398438 cx: 322.329406738281 cy: 251.370620727539 nerf: ros_args: intrinsic: [608.502868652344,608.542358398438, 322.329406738281, 251.370620727539] # K[0, 0], K[1, 1], K[0, 2], K[1, 2] 同理我修改了imu数据

我录了两个数据包第一个数据包vins的tracking运行没有问题而第二个包在只有墙面的时候报了错误目前我分析是因为没有特征点导致slam运行出错我现在重新录制了一个数据包再进行尝试

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2436979796, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNRR24EPO7X4XS4JZWTZ5HQDLAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZWHE3TSNZZGY. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

我现在的数据包vins运行没有问题但是在建图的时候出现了下面的内容请问是什么问题万分感谢 Segmentation fault (core dumped)

JIANG-CX commented 1 month ago

这个报错的原因有很多的，建议单步调试看具体是哪里出现的问题

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 3:09:56 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

我现在的数据包vins运行没有问题但是在建图的时候出现了下面的内容请问是什么问题万分感谢 Segmentation fault (core dumped)

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437057657, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNRCXCWPD6S2QR364HTZ5HVEJAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGA2TONRVG4. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

好的我现在研究一下

zfkdsg commented 1 month ago

好像是显存的问题我这次重新运行程序但是我一直在看nvidia-smi 0 NVIDIA GeForce RTX 3090 Off | 00000000:3B:00.0 Off | N/A | | 60% 59C P8 30W / 350W | 16455MiB / 24576MiB | 0% Default 最后又出现了其他的错误显示 torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 134.00 MiB (GPU 0; 23.69 GiB total capacity; 12.74 GiB already allocated; 5.00 MiB free; 14.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

我现在尝试减少一些内存需求：我决定先修改一下参数进行尝试：文件如下： decoder_specs: #编码方式配置文件 voxel_size: 0.1 # same as mapper_specs 体素大小 L: 4 # Number of levels
F_entry: 2 # Number of feature dimensions per entry log2_T: 19 # each level's hashmap_size = F_entry * (2*F_entry) b: 2.0 # each level's resolution = N_min (b**Level)

mapper_specs: start_frame: 0 #初始帧 end_frame: -1 #结束帧 N_rays_each: 4096 # mapping's sampling ray 每个的渲染个数 num_vertexes: 200000 #节点数字 inflate_margin_ratio: 0.1 # voxel_size: 0.1 #这里我们增大一下改称0.02 原始数据是 0.1 step_size: 0.1 #步长 num_iterations: 8
max_voxel_hit: 10 #最大击中点数量 final_iter: 0 mesh_res: 8
overlap_th: 0.8 kf_window_size: 8 #滑动窗口 kf_selection_method: "multiple_max_set_coverage" # "random” or “multiple_max_set_coverage” 最大选择帧方式 kf_selection_random_radio: 0.5 # random keyframe ratio insert_method: "intersection" # "naive" or "intersection" insert_ratio: 0.85 offset: 10 # used to make make the coordinate of each point positive 偏执 use_adaptive_ending: True # adaptive iteration

我准备减少 N_rays_each ：2048 num_vertexes:100000 来进行实验我现在的问题是还有哪些减少建图显存要求的方法谢谢

JIANG-CX commented 1 month ago

你在我提供的bag上会有显存问题吗？

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 3:31:32 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

好像是显存的问题我这次重新运行程序但是我一直在看nvidia-smi 0 NVIDIA GeForce RTX 3090 Off | 00000000:3B:00.0 Off | N/A | | 60% 59C P8 30W / 350W | 16455MiB / 24576MiB | 0% Default 最后又出现了其他的错误显示 torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 134.00 MiB (GPU 0; 23.69 GiB total capacity; 12.74 GiB already allocated; 5.00 MiB free; 14.30 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

我现在尝试减少一些内存需求：我决定先修改一下参数进行尝试：文件如下： decoder_specs: #编码方式配置文件 voxel_size: 0.1 # same as mapper_specs 体素大小 L: 4 # Number of levels F_entry: 2 # Number of feature dimensions per entry log2_T: 19 # each level's hashmap_size = F_entry (2F_entry) b: 2.0 # each level's resolution = N_min (bLevel)

mapper_specs: start_frame: 0 #初始帧 end_frame: -1 #结束帧 N_rays_each: 4096 # mapping's sampling ray 每个的渲染个数 num_vertexes: 200000 #节点数字 inflate_margin_ratio: 0.1 # voxel_size: 0.1 #这里我们增大一下改称0.02 原始数据是 0.1 step_size: 0.1 #步长 num_iterations: 8 max_voxel_hit: 10 #最大击中点数量 final_iter: 0 mesh_res: 8 overlap_th: 0.8 kf_window_size: 8 #滑动窗口 kf_selection_method: "multiple_max_set_coverage" # "random” or “multiple_max_set_coverage” 最大选择帧方式 kf_selection_random_radio: 0.5 # random keyframe ratio insert_method: "intersection" # "naive" or "intersection" insert_ratio: 0.85 offset: 10 # used to make make the coordinate of each point positive 偏执 use_adaptive_ending: True # adaptive iteration

我准备减少 N_rays_each ：2048 num_vertexes:100000 来进行实验我现在的问题是还有哪些减少建图显存要求的方法谢谢

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437093216, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNUAN6S7RZQKIMUGCFDZ5HXVJAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGA4TGMRRGY. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

没有我在原始数据上运行没有问题

zfkdsg commented 1 month ago

我的·显卡也是3090 显存是24GB

JIANG-CX commented 1 month ago

建议可以参考https://github.com/SYSU-STAR/H2-Mapping/issues/45修改关键帧存储，一般是关键帧存过多导致显存问题

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 3:37:36 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

现在是自己采集的数据不过也是室内小场景的环境

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437103814, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNWT3HK5LBGV3PDJ7OTZ5HYMBAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGEYDGOBRGQ. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

也就是说这个问题是关键帧太多了导致的论文中我记得关键帧的选择方式是与前一帧差别比较大的会选作关键帧
但是看完你给的这个参考后好像是说H3 mapping 解决了类似的问题我现在去尝试一下H3 mapping 感谢你的快速回复

JIANG-CX commented 1 month ago

当场景比较大的时候，比如在一个地方重复走，在h2mapping中关键帧还是会不断累积，在h3mapping中加了一个策略来删减。你可以先确定一下是不是关键帧存的太多导致的问题

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 4:11:05 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

也就是说这个问题是关键帧太多了导致的论文中我记得关键帧的选择方式是与前一帧差别比较大的会选作关键帧但是看完你给的这个参考后好像是说H3 mapping 解决了类似的问题我现在去尝试一下H3 mapping 感谢你的快速回复

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437165381, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNR5SQOR6ZMHKUZAMTTZ5H4JTAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGE3DKMZYGE. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

这个怎么确定是关键帧存的太多呢？？

JIANG-CX commented 1 month ago

你可以看一下存的关键帧的数量，看一下他占据的显存大小是不是过大

Get Outlook for iOShttps://aka.ms/o0ukef

From: zfkdsg @.> Sent: Friday, October 25, 2024 4:15:38 PM To: SYSU-STAR/H2-Mapping @.> Cc: JIANG Chenxing @.>; Comment @.> Subject: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

这个怎么确定是关键帧存的太多呢？？

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437174361, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNXSD4HK2PORKA53I23Z5H42VAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGE3TIMZWGE. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

好的我去看一下代码感谢回复

zfkdsg commented 1 month ago

关键帧的数量 507 idx: 621 我输出了关键帧的数量这里显示的是 507帧这里是不是会导致占据的显存太大了

JIANG-CX commented 1 month ago

比较偏多了，这边621帧就有507帧加入到关键帧池中，一般来说是由于深度相对比较稀疏，导致两帧之间所对应的voxel差较大，导致加入关键帧池的帧过多。最简单的方式是减小insert_ratio这个参数，也可以试一下h3mapping中的策略

发件人: zfkdsg @.> 发送时间: 2024年10月25日 16:38 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

关键帧的数量 507 idx: 621 我输出了关键帧的数量这里显示的是 507帧这里是不是会导致占据的显存太大了

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437218432, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNXUYWNNOXVI5XHYNKTZ5H7OVAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGIYTQNBTGI. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

好的好的感谢回复我现在准备两种方法都试一试第一就是安装h3 mapping 并运行第二就是重新采数据不来回走动一步到位然后在减小insert_ratio 在进行尝试

JIANG-CX commented 1 month ago

不过根据之前的issue，用435的效果不是很好，可以看一下相关的issue和回复

发件人: zfkdsg @.> 发送时间: 2024年10月25日 16:46 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

好的好的感谢回复我现在准备两种方法都试一试第一就是安装h3 mapping 并运行第二就是重新采数据不来回走动一步到位然后在减小insert_ratio 在进行尝试

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2437234235, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNUWCL55BJ4OL5UNKRTZ5IAM3AVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMZXGIZTIMRTGU. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

好的好的我现在先去尝试解决问题在看相机的问题

zfkdsg commented 1 month ago

您好我现在用来了L515想尝试采数据但是问题现在我使用D435i的相机驱动却检测不到设备而且我使用rs-enumerate-devices -c 仍然检测不到设备 realsense-viewer也是检测不到请问作者我的驱动就是https://github.com/IntelRealSense/realsense-ros这里的我现在想分析一下问题请问作者的驱动是这个吗

JIANG-CX commented 1 month ago

对的，就用的官方的ros包，然后realsense sdk用的2.5

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: zfkdsg @.> 发送时间: Monday, October 28, 2024 12:13:45 PM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

您好我现在用来了L515想尝试采数据但是问题现在我使用D435i的相机驱动却检测不到设备而且我使用rs-enumerate-devices -c 仍然检测不到设备 realsense-viewer也是检测不到请问作者我的驱动就是https://github.com/IntelRealSense/realsense-ros这里的https://github.com/IntelRealSense/realsense-ros%E8%BF%99%E9%87%8C%E7%9A%84 我现在想分析一下问题请问作者的驱动是这个吗

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2440513076, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNUALUNBKFQXF6ILX7TZ5W2XTAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBQGUYTGMBXGY. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

您好我想问一下你遇到过 rs-enumerate-devices：未找到命令的问题吗我现在L515可以驱起来了也有了数据但是 ·我现在想看内参显示找不到命令

JIANG-CX commented 1 month ago

没有诶

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: zfkdsg @.> 发送时间: Monday, October 28, 2024 3:30:17 PM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

您好我想问一下你遇到过 rs-enumerate-devices：未找到命令的问题吗我现在L515可以驱起来了也有了数据但是・我现在想看内参显示找不到命令

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2440748705, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNSOWZFHLGUWUIVOX2TZ5XRYTAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBQG42DQNZQGU. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

请问SDK2.0是不支持检测L515吗我按照官方网站安装的2.0 结果 rs-enumerate-devices No device detected. Is it plugged in? 驱动可以驱起来但是还是检测不到设备 realsense-viewer 也是没有

zfkdsg commented 1 month ago

解决了应该是sdk的安装问题

JIANG-CX commented 1 month ago

ok

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: zfkdsg @.> 发送时间: Monday, October 28, 2024 7:12:01 PM 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

解决了应该是sdk的安装问题

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2441288032, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNX4LMHWW4XEW2ZKWULZ5YLYDAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBRGI4DQMBTGI. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

作者你好我今天条通了自己的L515驱动相机并且成功的运行了代码不过效果还是欠佳我想和您交流一下：首先我的L515相机参数图片参数是 1280*720 因此我查找rs-enuserate-devices 输出如下： Intrinsic of "Color" / 1280x720 / {YUYV/RGB8/BGR8/RGBA8/BGRA8/Y16} Width: 1280 Height: 720 PPX: 641.458190917969 PPY: 366.066802978516 Fx: 909.705322265625 Fy: 910.015380859375 Distortion: Brown Conrady Coeffs: 0.158648923039436 -0.483447372913361 -0.000308293238049373 0.000581265368964523 0.420061677694321

Extrinsic from "Color" To "Gyro" : Rotation Matrix: 0.999992 0.00213505 0.00335176
-0.00204156 0.999615 -0.0276533
-0.00340952 0.0276462 0.999612

Translation Vector: -0.0114989820867777 0.00227184034883976 0.00521943671628833

因此我更改了我的slam参数以及 nerf参数分别是： uav_nerf.yaml distortion_parameters: k1: 0.158648923039436 k2: -0.483447372913361 p1: 0.000581265368964523 p2: 0.420061677694321 projection_parameters: fx: 909.705322265625 fy: 910.015380859375 cx: 641.458190917969 cy: 366.066802978516

extrinsicRotation: !!opencv-matrix rows: 3 cols: 3 dt: d data: [ 0.999992 , 0.00213505 ,0.00335176, -0.00204156 , 0.999615 ,-0.0276533, -0.00340952 , 0.0276462 ,0.999612]

Translation from camera frame to imu frame, imu^T_cam

extrinsicTranslation: !!opencv-matrix rows: 3 cols: 1 dt: d data: [-0.0114989820867777 , 0.00227184034883976 ,0.00521943671628833 ] 以及我的 config/realsense/realsense.yaml intrinsic: [909.705322265625, 910.015380859375, 641.458190917969, 366.066802978516] # K[0, 0], K[1, 1], K[0, 2], K[1, 2]

之后我便采数据进行实验发现效果还是欠佳我觉得明天多采几组数据进行尝试

不过我今天在采数据的过程中发现一些问题,例如： 1.realsense 的alien-depth对其深度在固定不变的时候可能会出现闪烁我采了数据并输出了深度发现有一些误差，这是不是意味着我们在采数据的时候最好不要固定不变

2.其次我在采数据的时候发现光照好像对最终深度估计有影响我们采数据的时候是不是最好在没有反光的地方进行数据采集？还是说对于L515+H2 mapping来说有些反光也没有影响

JIANG-CX commented 1 month ago

先回答后面的两个问题：

看到的闪烁应该只是因为fov中最小最大深度值发生变化导致的深度可视化时候颜色的变化，对算法没有影响
算法暂时不支持对反光的处理

其次回答参数文件的问题：请检查uav_nerf.yaml中 image_width，image_height是否也进行了更改

最后回答实验问题：在使用realsense实验时，首先确保tracking正常运行，pose的估计是稳定的，比如如果对着一个没纹理的区域，tracking会无法正常运行。你可以在tracking初始化结束后先测试一下静置realsense，运行mapping一段时间，然后结束，看是否从能得到正常的mesh文件，来检查是否有设置不对的地方。

zfkdsg commented 1 month ago

祖宗哦和你好请问我在采数据的时候对其深度部分发现有很多缺失的地方请问这种会对最后的结果有很大的影响吗你在采数据的时候有遇到过这个问题吗

JIANG-CX commented 1 month ago

L515对有些物体，比如黑色物体的深度估计会失效。目前的算法中，这些一直没有深度的地方重建后会有空洞。

zfkdsg commented 1 month ago

作者你好我想问一下如果说我开始录了一个物体结果这个物体在之后的图片中又出现了会不会对实验结果造成影响

zfkdsg commented 1 month ago

还有就是我在运行slam的时候出现了 Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed 0.578316 | nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan 类似于这种这种是不是就代表着位姿飘了

JIANG-CX commented 1 month ago

如果位姿估计保持准确，不会有影响，就跟replica数据一样。如果pose不准确，会造成影响

发件人: zfkdsg @.> 发送时间: 2024年10月31日 17:22 收件人: SYSU-STAR/H2-Mapping @.> 抄送: JIANG Chenxing @.>; Comment @.> 主题: Re: [SYSU-STAR/H2-Mapping] 运行自己的数据报错 (Issue #48)

作者你好我想问一下如果说我开始录了一个物体结果这个物体在之后的图片中又出现了会不会对实验结果造成影响

― Reply to this email directly, view it on GitHubhttps://github.com/SYSU-STAR/H2-Mapping/issues/48#issuecomment-2449398658, or unsubscribehttps://github.com/notifications/unsubscribe-auth/APMKKNRPPPPYUNAACDNOZB3Z6HZELAVCNFSM6AAAAABQQELXNSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBZGM4TQNRVHA. You are receiving this because you commented.Message ID: @.***>

zfkdsg commented 1 month ago

您好我今天有录制了一些数据包再进行位姿估计的时候出现了： Error in evaluating the ResidualBlock.

There are two possible reasons. Either the CostFunction did not evaluate and fill all
residual and jacobians that were requested or there was a non-finite value (nan/infinite) generated during the or jacobian computation.

Residual Block size: 4 parameter blocks x 27 residuals

For each parameter block, the value of the parameters are printed in the first column
and the value of the jacobian under the corresponding residual. If a ParameterBlock was held constant then the corresponding jacobian is printed as 'Not Computed'. If an entry of the Jacobian/residual array was requested but was not written to by user code, it is indicated by 'Uninitialized'. This is an error. Residuals or Jacobian values evaluating to Inf or NaN is also an error.

Residuals: nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan nan -nan

Parameter Block 0, size: 7

-0.011499 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
0.00227184 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
0.00521944 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
0.0138262 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
0.00169049 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
-0.00104425 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed
0.999902 | Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed Not Computed

Parameter Block 1, size: 7

0.268726 |          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan          nan 

请问这是什么问题 感谢回答

SYSU-STAR / H2-Mapping

运行自己的数据报错 #48

projection_parameters:

fx: 456.377

fy: 607.001

cx: 321.747

cy: 251.001

Translation from camera frame to imu frame, imu^T_cam