visualization problem

[wangjuansan]

Hi, I found a visualization problem when I follow your docs/INFER.md.

1. python test.py --cfg_file cfgs/scannet_models/spconv_clip_base15_caption_adamw.yaml --ckpt output/scannet_models/spconv_clip_base15_caption/exp_tag/ckpt/checkpoint_epoch_128_6488.pth --save_results semantic,instance
2. python visual_utils/visualize_indoor.py

Traceback (most recent call last):
  File "visual_utils/visualize_indoor.py", line 217, in <module>
    rooms = sorted(os.listdir(opt.prediction_path + '/pred_instance'))
FileNotFoundError: [Errno 2] No such file or directory: './results/pred_instance'

Then I got into eval_utils.py, https://github.com/CVMI-Lab/PLA/blob/3d8494b9499c9f7f2b1ed869fd7f57c960042a1a/tools/eval_utils/eval_utils.py#L85

The value of parameter eval_output_dir is /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488. So I'm confused that when I run test.py, set --save_results semantic,instance, it should generate files ending in py and pth in the PLA/data/scannetv2/val/ and PLA/data/scannetv2/val_pth/ directory, but it not.

Or am I misunderstanding the steps of visualization? Looking forward to your reply.Any assistant will be appreciated.

[Dingry]

Hi, the results will be saved into the eval_output_dir which is /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488 in your case. You should find 'pred_instance' and 'semantic_pred' directories under your 'eval_output_dir'. For instance evaluation, the saving code is here. https://github.com/CVMI-Lab/PLA/blob/3d8494b9499c9f7f2b1ed869fd7f57c960042a1a/tools/eval_utils/eval_utils.py#L275-L288

[wangjuansan]

Thank you for your reply, but there were no eval_output_dir in /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488 after running python test.py --cfg_file cfgs/scannet_models/spconv_clip_base15_caption_adamw.yaml --ckpt output/scannet_models/spconv_clip_base15_caption/exp_tag/ckpt/checkpoint_epoch_128_6488.pth --save_results semantic,instance.

[Dingry]

Hi, could you please provide a list of the contents in '/PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488'?

[wangjuansan]

Several txt file(eg. log_eval_20230914-141741.txt) in val/default/. That's all.

[Dingry]

Hello, based on the information you provided, the results are expected to be stored in the directory /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488/val/default/.

And the eval_output_dir in the code that saves the results should also be set to this path, rather than just /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488.

To locate the exact location of the eval_output_dir in the code, you can step in the code by yourself. Alternatively, you can use the find command to search for the saved results.

Here comes my example:

[wangjuansan]

Hi, thank you for your attention, yes the eval_output_dir is /PLA/output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488/val/default/.

The previous command I run python test.py --cfg_file cfgs/scannet_models/spconv_clip_base15_caption_adamw.yaml --ckpt output/scannet_models/spconv_clip_base15_caption/exp_tag/ckpt/checkpoint_epoch_128_6488.pth --save_results semantic,instance is wrong.

It should be python test.py --cfg_file cfgs/scannet_models/spconv_clip_base15_caption_adamw.yaml --ckpt output/scannet_models/spconv_clip_base15_caption/exp_tag/ckpt/checkpoint_epoch_128_6488.pth --save_results semantic & python test.py --cfg_file cfgs/scannet_models/inst/spconv_clip_base13_caption_adamw.yaml --ckpt output/scannet_models/spconv_clip_base15_caption/exp_tag/ckpt/checkpoint_epoch_128_6488.pth --save_results instance.

--save_results semantic,instance cannot be recognized.

BTW, I'm a little confused with the yaml file name. I notice that the base_class_idx in cfgs/scannet_models/inst/spconv_clip_base13_caption_adamw.yaml & cfgs/scannet_models/spconv_clip_base15_caption_adamw.yaml is same. I mean if the file name base** is base on base_class_idx?

And when I run python tools/visual_utils/visualize_indoor.py --prediction_path output/scannet_models/spconv_clip_base15_caption_adamw/default/eval/epoch_6488/val/default/ --task semantic_pred, I got

0 scene0011_00.npy
Traceback (most recent call last):
  File "tools/visual_utils/visualize_indoor.py", line 234, in <module>
    vis(opt)
  File "tools/visual_utils/visualize_indoor.py", line 165, in vis
    data_dict = get_coords_color(opt)
  File "tools/visual_utils/visualize_indoor.py", line 61, in get_coords_color
    data_dict = get_input(opt)
  File "tools/visual_utils/visualize_indoor.py", line 27, in get_input
    xyz, rgb, alpha, face_indices = read_ply(input_file)
  File "/var/autofs/home/hale/usrs/wangjuan/code/PLA/tools/visual_utils/indoor_utils/ply_utils.py", line 6, in read_ply
    plydata = PlyData.read(path)
  File "/home/usrs/wangjuan/anaconda3/envs/PLA/lib/python3.7/site-packages/plyfile.py", line 498, in read
    (must_close, stream) = _open_stream(stream, 'read')
  File "/home/usrs/wangjuan/anaconda3/envs/PLA/lib/python3.7/site-packages/plyfile.py", line 657, in _open_stream
    return (True, open(stream, read_or_write[0] + 'b'))
FileNotFoundError: [Errno 2] No such file or directory: '../../data/scannetv2/val/scene0011_00_vh_clean_2.ply'

https://github.com/CVMI-Lab/PLA/blob/3d8494b9499c9f7f2b1ed869fd7f57c960042a1a/tools/visual_utils/visualize_indoor.py#L19-L23

Is the visualization code not up to date?

[Dingry]

Maybe the command should be updated to --save_results instance semantic. We will look at it later. If you do not need to write ply file with mesh when visualization, you can comment out line 25, 31, 32 and modify the line 28 into xyz, rgb, label, inst_label, *_ = torch.load(label_file) in https://github.com/CVMI-Lab/PLA/blob/3d8494b9499c9f7f2b1ed869fd7f57c960042a1a/tools/visual_utils/visualize_indoor.py#L25C4-L32.

[wangjuansan]

Hi, thank you for your reply. Maybe I didn't express myself clearly, what I meant was that when I run python tools/visual_utils/visualize_indoor.py I got FileNotFoundError: [Errno 2] No such file or directory: '../../data/scannetv2/val/scene0011_00_vh_clean_2.ply'.

In fact, the file named scene0011_00_vh_clean_2.ply is indeed not in the data I downloaded directly by following your data processing file docs/DATASET.md.

And the visualize_indoor.py requires a file with a "_vh_clean_2.ply" suffix, so that's causing the file not found error. So I am wondering if the file visualize_indoor.py needs to be modified?

[Dingry]

The PLY file is not necessary. If you do not need to visualize the mesh, you can disregard this file based on my previous instructions. However, if you wish to visualize the mesh with better quality, you can download the PLY file from the ScanNet website. We are unable to distribute these files due to certain conditions. We will provide a clearer version of the visualization code in the future.

[wangjuansan]

Thank you for your reply. Looking forward to your clearer version of the visualization code.