Scannet.py

[zjw102]

Hello author, in the scannet.py file combine_pred_sem_path = os.path.join(DATASET_ROOT_DIR, 'tag2text_pred_dino', scene_name, 'combine' + frame_name + '.npy'), where there is no combine_pred_sem_path this file to the path, Am I missing something?

[Zhimin-C]

Just run python pred_seg.py

[zjw102]

Thanks for the reply, but I have a new problem, in the scannet.py file caption_feat = os.path.join(DATASET_ROOT_DIR, 'caption', scene_name, frame_name + '_caption.npy'), but I only get caption.txt file, caption.npy is a typo.,If so_caption.txt how do you load it?

[Zhimin-C]

You can generate it with tag2text_SAM.py, see line 530 to 531. with open(text_base_path + '/' + image_name[:-4] + '_caption.txt', 'w') as f: f.write(str(caption))

[zjw102]

No, I mean I've already got _caption.txt', but in the scannet.py line 240 caption_feat = os.path.join(DATASET_ROOT_DIR, 'caption', scene_name, frame_name + '_caption.npy') shows caption.npy instead of caption.txt, is the problem when I replace _caption.npy with _caption.txt but line 252 feat_2D = np.load(feat_2D) Can't load, looking forward to your reply

[Zhimin-C]

It appears that I did not upload the Python file needed to generate _caption.npy, and I cannot locate it at the moment. I will attempt to rewrite it. The _caption.npy file stores the representations extracted from the CLIP text-encoder using the captions in _caption.txt. You can directly use the CLIP encoder to generate this file. For guidance on generating _caption.npy from _caption.txt, you can refer to the Scene_2D_feats.py file.

[zjw102]

Thank you for your reply

[zjw102]

I tried to generate _caption.npy with the CLIP text encoder, but it doesn't feel right, I look forward to you uploading the Python file needed to generate _caption.npy as soon as possible, thank you!

[Zhimin-C]

Sorry for the late reply. I have updated the tag2text_SAM.py file to generate _caption.npy (see line 544). Now it should work.