search

hustvl / Symphonies

[CVPR 2024] Symphonies (Scene-from-Insts): Symphonize 3D Semantic Scene Completion with Contextual Instance Queries
https://arxiv.org/abs/2306.15670
MIT License
168 stars 6 forks source link

may i know how to run a test set #8

Closed dzrdzrdzr closed 11 months ago

npurson commented 11 months ago

I assume you are looking to evaluate on the validation set using a pre-trained checkpoint. I think you can achieve this by the followings:

  1. Modify tools/train.py. Specifically, replace https://github.com/hustvl/Symphonies/blob/f74b9d4ae15a9b11c6d692224053aa882e261076/tools/train.py#L19 with trainer.test(dls[1]) referring to https://lightning.ai/docs/pytorch/stable/common/trainer.html#testing.
  2. Load the checkpoint referring to https://github.com/hustvl/Symphonies/blob/f74b9d4ae15a9b11c6d692224053aa882e261076/tools/generate_outputs.py#L23

These steps should help you achieve your goal. Please don’t hesitate to reach out if you encounter any issues or have further questions.

dzrdzrdzr commented 11 months ago

well, this is validation set which seems only ran the eighth dataset. But how to run 11-20

npurson commented 11 months ago

As indicated in the README, execute the command python tools/test.py +ckpt_path=...

I would like to clarify that the test set doesn't provide labels for local evaluation. You may need to generate the predictions using this script and submit them to the test server at https://codalab.lisn.upsaclay.fr/competitions/7170.

dzrdzrdzr commented 11 months ago

thanks that's what needed