Questions on Eval/Dataset

[YoonjungChoi]

Hello. I am really new in this field of SIDP. I am trying to understanding your architecture.

As I understanding, raw rgb images are located under the 'kitti_raw' folder and other ground truth images are located under the 'gt' folder.

When I tried your pre-trained "KITTI EIGEN" model,

$ python vadepthnet/eval.py configs/yoon_arguments_eval_kittieigen.txt

Q. Do you have any ideas why it cannot read files and show "0 eval samples"?

/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torchvision/image.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
== Total number of parameters: 263110761
== Total number of learning parameters: 263110761
== Model Initialized
== Loading checkpoint 'ckpts/vadepthnet_eigen.pth'
== Loaded checkpoint 'ckpts/vadepthnet_eigen.pth'
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 697/697 [00:30<00:00, 22.61it/s]
Computing errors for 0 eval samples , post_process:  False
  silog, abs_rel,   log10,     rms,  sq_rel, log_rms,      d1,      d2,      d3
    nan,     nan,     nan,     nan,     nan,     nan,     nan,     nan,     nan

Q. When I tried to eval with kitti_official_valid.txt, I got issue depth_image s

depth_path = os.path.join(gt_path, "./" + sample_path.split()[1])
                                              ~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

So I added depth_image paths in the filed like this

(newDepth) [013907062@g5 VA-DepthNet]$ python vadepthnet/eval.py configs/yoon_arguments_eval_kittieigen.txt
/home/013907062/.conda/envs/newDepth/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
== Total number of parameters: 263110761
== Total number of learning parameters: 263110761
== Model Initialized
== Loading checkpoint 'ckpts/vadepthnet_eigen.pth'
== Loaded checkpoint 'ckpts/vadepthnet_eigen.pth'
100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [07:27<00:00,  2.24it/s]
Computing errors for 1000 eval samples , post_process:  False
  silog, abs_rel,   log10,     rms,  sq_rel, log_rms,      d1,      d2,      d3
 6.5207,  0.0461,  0.0198,  1.9626,  0.1426,  0.0714,  0.9802,  0.9967,  0.9991

Am I doing correctly? I wonder how can I check test set(without ground truth) from KITTI sites as well.

Q Where is located results of inference? Where can I look at output results?

[cnexah]

Hi, thank you for your interest! Q1: I suggest to follow the NeWCRFs (https://github.com/aliyun/NeWCRFs/tree/master) to prepare the data, and use their dataloader. Q2: kitti_official_valid.txt has different train/val splits from the eigen. I suggest to train the network with the corresponding training set. To evaluate on the test set, you have to sign up and submit at KITTI website (https://www.cvlibs.net/datasets/kitti/eval_depth.php?benchmark=depth_prediction). Q3: Please use test.py in NeWCRFs to save the prediction results.

[YoonjungChoi]

I am appreciated for your help!

[guwenxiang1]

I encountered a similar issue as you did. I cropped a certain dataset to match the size of the KITTI dataset and then executed the command "python eval.py". The specific results are as follows: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 305/305 [00:53<00:00, 5.71it/s] Computing errors for 305 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan

and, 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 240/240 [00:05<00:00, 43.40it/s] Computing errors for 0 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan

Is there any way to solve it?

[YoonjungChoi]

At that time, I didn't know the data folder structure of Eigen Split. I just combined train and test folders of KITTI official split set. Because I used "data_splits/eigen_test_files_with_gt.txt" and data is loaded that file. You can check your data path or something.

[cnexah]

I encountered a similar issue as you did. I cropped a certain dataset to match the size of the KITTI dataset and then executed the command "python eval.py". The specific results are as follows: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 305/305 [00:53<00:00, 5.71it/s] Computing errors for 305 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan

and, 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 240/240 [00:05<00:00, 43.40it/s] Computing errors for 0 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan

Is there any way to solve it?

Dear guwenxiang1,

Sorry for the late reply, do you still need any help?

[guwenxiang1]

Thank you for your attention. The problem has been resolved. ^-^

[Wujinnanqqqqqq]

I encountered a similar issue as you did. I cropped a certain dataset to match the size of the KITTI dataset and then executed the command "python eval.py". The specific results are as follows: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 305/305 [00:53<00:00, 5.71it/s] Computing errors for 305 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan and, 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 240/240 [00:05<00:00, 43.40it/s] Computing errors for 0 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan Is there any way to solve it?

Dear guwenxiang1,

Sorry for the late reply, do you still need any help?

hello i need your help

[Wujinnanqqqqqq]

Hello. I am really new in this field of SIDP. I am trying to understanding your architecture.

As I understanding, raw rgb images are located under the 'kitti_raw' folder and other ground truth images are located under the 'gt' folder.

When I tried your pre-trained "KITTI EIGEN" model,

$ python vadepthnet/eval.py configs/yoon_arguments_eval_kittieigen.txt

Q. Do you have any ideas why it cannot read files and show "0 eval samples"?

/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torchvision/io/image.py:13: UserWarning: Failed to load image Python extension: '/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torchvision/image.so: undefined symbol: _ZN5torch3jit17parseSchemaOrNameERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEE'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
/home/013907062/.conda/envs/depthEst/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
== Total number of parameters: 263110761
== Total number of learning parameters: 263110761
== Model Initialized
== Loading checkpoint 'ckpts/vadepthnet_eigen.pth'
== Loaded checkpoint 'ckpts/vadepthnet_eigen.pth'
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 697/697 [00:30<00:00, 22.61it/s]
Computing errors for 0 eval samples , post_process:  False
  silog, abs_rel,   log10,     rms,  sq_rel, log_rms,      d1,      d2,      d3
    nan,     nan,     nan,     nan,     nan,     nan,     nan,     nan,     nan

Q. When I tried to eval with kitti_official_valid.txt, I got issue depth_image s

depth_path = os.path.join(gt_path, "./" + sample_path.split()[1])
                                              ~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

So I added depth_image paths in the filed like this

(newDepth) [013907062@g5 VA-DepthNet]$ python vadepthnet/eval.py configs/yoon_arguments_eval_kittieigen.txt
/home/013907062/.conda/envs/newDepth/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
  return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
== Total number of parameters: 263110761
== Total number of learning parameters: 263110761
== Model Initialized
== Loading checkpoint 'ckpts/vadepthnet_eigen.pth'
== Loaded checkpoint 'ckpts/vadepthnet_eigen.pth'
100%|██████████████████████████████████████████████████████████████████████████████████████████████| 1000/1000 [07:27<00:00,  2.24it/s]
Computing errors for 1000 eval samples , post_process:  False
  silog, abs_rel,   log10,     rms,  sq_rel, log_rms,      d1,      d2,      d3
 6.5207,  0.0461,  0.0198,  1.9626,  0.1426,  0.0714,  0.9802,  0.9967,  0.9991

Am I doing correctly? I wonder how can I check test set(without ground truth) from KITTI sites as well.

Q Where is located results of inference? Where can I look at output results?

I would like to ask how you resolved the issue after this step, as the link you previously posted has become invalid

[guwenxiang1]

I encountered a similar issue as you did. I cropped a certain dataset to match the size of the KITTI dataset and then executed the command "python eval.py". The specific results are as follows: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 305/305 [00:53<00:00, 5.71it/s] Computing errors for 305 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan and, 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 240/240 [00:05<00:00, 43.40it/s] Computing errors for 0 eval samples , post_process: False silog, abs_rel, log10, rms, sq_rel, log_rms, d1, d2, d3 nan, nan, nan, nan, nan, nan, nan, nan, nan Is there any way to solve it?

Dear guwenxiang1,

Sorry for the late reply, do you still need any help?

Thank you, I don't have any more questions.