zlccccc / 3DVL_Codebase

[CVPR2022 Oral] 3DJCG: A Unified Framework for Joint Dense Captioning and Visual Grounding on 3D Point Clouds
Other
51 stars 5 forks source link

Instruction for SCAN2CAD dataset. #2

Closed sunanhe closed 2 years ago

sunanhe commented 2 years ago

Hi, I'm really interested in this work. But I cannot follow the Data preparation. It seems lack of the instruction for SCAN2CAD dataset.

zlccccc commented 2 years ago

Our data preprocessing strategy is the same as Scan2Cap. In addition, our training process should be independent of Scan2Cad. We will update the code as soon as possible. You may refer to the official preprocessing codes in Scan2Cap. https://github.com/daveredrum/Scan2Cap

sunanhe commented 2 years ago

Thanks for your timely reply. I have a new problem.

Traceback (most recent call last):
  File "scripts/joint_scripts/train_3djcg.py", line 503, in <module>
    train(args)
  File "scripts/joint_scripts/train_3djcg.py", line 433, in train
    solver, num_params, root = get_solver(args, dataset, dataloader)
  File "scripts/joint_scripts/train_3djcg.py", line 176, in get_solver
    model = get_model(args, dataset["train"], device)
  File "scripts/joint_scripts/train_3djcg.py", line 88, in get_model
    dataset_config=DC
  File "3DJCG/models/jointnet/jointnet.py", line 65, in __init__
    self.caption = SceneCaptionModule(vocabulary, embeddings, emb_size, 128, caption_hidden_size, num_proposal)
  File "3DJCG/models/capnet/caption_module.py", line 83, in __init__
    raise NotImplementedError()
NotImplementedError 
zlccccc commented 2 years ago

We use the model: _https://github.com/zlccccc/3DJCG/blob/main/models/capnet/caption_module.py#L86_ For the training scripts, please use: _python scripts/joint_scripts/train_3djcg.py --use_multiview --use_normal --use_topdown --num_graph_steps 0 --num_locals 20 --batch_size 10 --epoch 200 --gpu 2 --verbose 50 --val_step 1000 --lang_num_max 8 --coslr --lr 0.002 --num_groundepoch 150 --tag 3djcg We will update the code and README as soon as possible.

Zhang-Jing-Xuan commented 2 years ago

Hi, I'm really interested in this work. But where is ScanRefer_filtered_organized.json? I can't find it in ScanRefer dataset.

zlccccc commented 2 years ago

We are using the original ScanRefer dataset, the downloaded file name is ScanRefer_filtered_organized.json. Please follow the official link https://github.com/daveredrum/ScanRefer to fill out this form. Once your request is accepted, you will receive an email with the download link.

Zhang-Jing-Xuan commented 2 years ago

OK, I can run it now.

zlccccc commented 2 years ago

We did a double-check to make sure that the original ScanRefer & Scan2Cap datasets were used for both training and testing in the paper. It is possible that we changed the dataset name when we downloaded it earlier. Please change "ScanRefer_filtered_organized.json" to "ScanRefer_filtered.json" in the code.

Zhang-Jing-Xuan commented 2 years ago

ScanRefer_filtered_organized.json can be obtained by [Code].