RGB or RGBD?

[Gan868611]

hi, I intended to to handle my project with this repo , but can i confirm that is this repo use solely RGB images only (without depth image) or is it RGB-D images(with requirement of depth map)? If it's compatible with RGB images only, what can i do to modify the code to achieve this?

[DecaYale]

Yes, we only take RGB images as inputs during inference without depth images.

[Gan868611]

what about during training? does it also use RGB only? when i run bash scripts/eval.sh ,i found that it used dataset from this directory which has depth images inside, so i am a bit confused

[DecaYale]

Like many other RGB-based methods, we have the CAD object model available. These depth images should be our rendered depth maps according to the object model and GT poses for training.

[Gan868611]

i see, can i know is there any code provided for generating all the essential file(data preprocess) like the depth images from the CAD file and GT poses I have?

[DecaYale]

We follow deepim and pvnet for data generation. You could refer to their official github repo for more details. Thx.

[Gan868611]

i see, can i know which code can help me to generate the depth from my CAD models and GT poses?

[DecaYale]

The following links could be helpful. We strictly follow their instructions for data preparation. Thx.
https://github.com/liyi14/mx-DeepIM/blob/master/prepare_data.md https://github.com/zju3dv/pvnet-rendering

[Arrebol2020]

Like many other RGB-based methods, we have the CAD object model available. These depth images should be our rendered depth maps according to the object model and GT poses for training.

Hello, in linemode_dataset.py, line 309: depth = os.path.join(self.root_paths[dataset_idx], info[frame_idx]['depth_gt_observed_path']) it use the gt poses in both training and eval, so I'm not sure this is rgb only.

[Arrebol2020]

When I replace it with: depth = np.zeros((rgb.shape[:2])) the network output the same result. I'm sure it's rgb method, I'm so sorry!