lm_imgn 数据库的问题

[Luckyangle1232]

我从GDRNet中下载的lm_imgn 中没有code_images模块，作者能否提供一个完整的lm_imgn库

[Luckyangle1232]

还有一个问题，我设置不加载模型，还是会出现找不到'experiments/checkpoints/init_gnn2_hrnetw18_npt512_lm/'的问题，方便也提供一下这个目录下的模型吗

[RuyiLian]

1. about code_image:

   • It is not used at all but I forget to delete the corresponding part in the dataset codes.
   • I think simply removing the dataset codes involving code_image should fix the problem. I will also update the codes in this repo.
   • If you download lm_imgn from GDR-Net, you just need to further run get_fps_points.py to generate the sampled dense keypoints (or use the ones provided in checkerpose/datasets/BOP_DATASETS/lm/fps_202212). Besides, you may need to download VOCdevkit images for replacing the background of LM images (see details in GDR-Net). I generate the ground truth binary codes on the fly during training instead of storing GT files and loading.

2. about --load_checkpoint

   • This parameter is used to load an existing checkpoint when the training process is interrupted.
   • Even if it is False, the training process requires an init model generated from pretrain.py or pretrain_lm.py. The pretrain scripts update the weights of low level network layers (maybe a more proper name is the first stage of the two-stage training process).
   • If you have run pretrain scripts but still have path related problem, please try to move the pretrained files or modify the paths in config files.
   • Otherwise, you could run the pretrain scripts to generate the required init models. It only requires a few hours even on a small GPU, so I do not upload the init models.

---

1. 这个code_images并没有用到，你可以尝试把相关的代码删除看看能不能解决问题。
2. 训练文件需要一个用pretrain.py 或者 pretrain_lm.py 得到的init model。

[RuyiLian]

I have modified the LM dataset codes and tested by myself. The newest version is uploaded. Please check it, thanks!

[Luckyangle1232]

I have modified the LM dataset codes and tested by myself. The newest version is uploaded. Please check it, thanks!

Thanks for your reply! The code is workable. But I meet a new problem. 2024-03-31 11:26:21.296922 hr18GNN2_res6_gnn3Skip_mlpQuery_lm iteration_step: 0 train_stage: 3 loss_roi_bit: 0.004630818031728268 loss_proj_bit_x: 0.3828274607658386 loss_proj_bit_y: 0.38404062390327454 loss_seg_visib: 0.49522408843040466 loss_seg_full: 0.5205239057540894 loss: 1.7872469425201416 time: 1.340477s est: 44.682h 0%| | 0/1679 [00:00<?, ?it/s]test dataset 0%| | 0/1679 [00:00<?, ?it/s] Traceback (most recent call last): File "/media/zhujq/deepl/6Dpose/CheckerPose-main/checkerpose/train_lm.py", line 393, in main(configs) File "/media/zhujq/deepl/6Dpose/CheckerPose-main/checkerpose/train_lm.py", line 341, in main test_pipeline_lm(net, test_loader, writer, iteration_step, configs, lm_p3d_normed, lm_p3d_xyz, File "/media/zhujq/deepl/6Dpose/CheckerPose-main/checkerpose/test_network_with_test_data.py", line 631, in test_pipeline_lm adx_error = Calculate_Pose_Error_Main(r_GT, t_GT, pred_rot, pred_trans, vertices_dict[obj_id]) KeyError: 1

[RuyiLian]

I just rerun the codes but I did not encounter this issue. Based on the provided information, the training process is fine but the periodic testing is problematic. The error message looks like key error of the vertices_dict, which is created as https://github.com/RuyiLian/CheckerPose/blob/fb725bed2a9eb6c3646c476c017d9ad00aa63a28/checkerpose/train_lm.py#L109-L112 so it should accept 1 as a key. Maybe you could try to print the dict keys and see what happens.

[Luckyangle1232]

I just rerun the codes but I did not encounter this issue. Based on the provided information, the training process is fine but the periodic testing is problematic. The error message looks like key error of the vertices_dict, which is created as

https://github.com/RuyiLian/CheckerPose/blob/fb725bed2a9eb6c3646c476c017d9ad00aa63a28/checkerpose/train_lm.py#L109-L112

so it should accept 1 as a key. Maybe you could try to print the dict keys and see what happens.

把这段代码注释解开就可以正常运行了，感谢您的帮助！！！！

[Luckyangle1232]

I just rerun the codes but I did not encounter this issue. Based on the provided information, the training process is fine but the periodic testing is problematic. The error message looks like key error of the vertices_dict, which is created as https://github.com/RuyiLian/CheckerPose/blob/fb725bed2a9eb6c3646c476c017d9ad00aa63a28/checkerpose/train_lm.py#L109-L112

so it should accept 1 as a key. Maybe you could try to print the dict keys and see what happens.

把这段代码注释解开就可以正常运行了，感谢您的帮助！！！！

另外还有一个问题，我在没有安装依赖库Progressive-X下现在代码是可以正常使用的，我可以问一下这个库有什么作用吗，或者他在哪里发挥作用

[RuyiLian]

我们的方法是个两阶段的方法，第一阶段是用网络估计3D-2D对应关系（通过keypoint localization实现），第二阶段是用现有的PnP solver求解pose。在上面 train_lm.py为了节省时间测试过程中的PnP solver是用的opencv的，Progressive-X是另一种PnP solver，速度慢但是一般来说估计的更准确，在 test_lm.py 里可以选择调用 progressive-x (命令行参数 --use_progressivex）。

[Luckyangle1232]

我们的方法是个两阶段的方法，第一阶段是用网络估计3D-2D对应关系（通过keypoint localization实现），第二阶段是用现有的PnP solver求解pose。在上面 train_lm.py为了节省时间测试过程中的PnP solver是用的opencv的，Progressive-X是另一种PnP solver，速度慢但是一般来说估计的更准确，在 test_lm.py 里可以选择调用 progressive-x (命令行参数 --use_progressivex）。

明白了，大神，再问一个，你这是使用的纯RGB图出的效果，我想测试一下您的代码在RGB-D图下的性能，请请问您的LM和YCB数据集读取代码有没有读取点云的或者有关参考，我自己写了一个读取的稍微有点问题

[RuyiLian]

Maybe you could check the codes of ZebraPose. The authors provide support for ICP refinement based on depth.