search

Jumpat / SegmentAnythingin3D

Segment Anything in 3D with NeRFs (NeurIPS 2023)
Apache License 2.0
862 stars 53 forks source link

About quantative result testing #57

Open ZHAOBenyun opened 8 months ago

ZHAOBenyun commented 8 months ago

Dear authors, thank you very much for open source this amazing work.

I am a beginner in 3D rendering and SAM. I would like to learn from you how to test on the datasets of NVOS, spin-nerf and Replica.

Would you release the relevant test codes? It may be a great help.

Jumpat commented 8 months ago

Hello! Thanks for your interest! The test code is not compatible with this GUI version currently. We will try to modify the code and provide it recently.

ZHAOBenyun commented 8 months ago

Thank you so much for your reply.

That is a great great new!!! Thank you again. Looking forward to your updates about the testing codes, that will be a big help for me to learn NeRF and SAM.

Zanue commented 7 months ago

We update a python script here for evaluating IoU, and modify lib/render_utils.py to save rendered masks when using --dump_images.

As an interactive segment method, in the test stage, we choose the first frame in the train set to perform usr-specific segment, train our model at the rest images of the train set, and then save the masks rendered from test views. The masks are used to calculate iou with gts in lib/evaluate.py.

ZHAOBenyun commented 7 months ago

Thank you so much!!

I will try as soon as possible and give some feedback to you, thanks again.

ZHAOBenyun commented 7 months ago

Hi, may I ask about how to load the replica spin-nerf dataset like config/llff, lerf, nerf-360? Would you share the config files?

Zanue commented 6 months ago

lerf/nerf unbounded configs are general for outdoor scenes. You can simply modify them to get your configs.

Note if the camera poses in your dataset use LLFF/OpenGL format, we have inverse_y=False in the config in default. If the camera poses use OpenCV format, you should set inverse_y=True.