search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
8.87k stars 1.18k forks source link

Integrate SAM model with Nerfstudio #3248

Open abrahamezzeddine opened 1 week ago

abrahamezzeddine commented 1 week ago

Hello,

I am wondering if it’s possible to integrate SAM masks for each frame with splatfacto, so that each point perhaps also gets an additional masking semantic attribute based on mask?

Any suggestion would be very helpful.

Thank you.

TamirCohen commented 1 day ago

I think it might be cool to add implementation of https://feature-3dgs.github.io/ (maybe as an external method). Then you can add SAM and any other feature field easly.

I am not nerfstudio developer / maintainer, I just think it is a useful feature. I can help implement it.

abrahamezzeddine commented 1 day ago

I think it might be cool to add implementation of https://feature-3dgs.github.io/ (maybe as an external method). Then you can add SAM and any other feature field easly.

I am not nerfstudio developer / maintainer, I just think it is a useful feature. I can help implement it.

Would be great! I would be happy to test it out and provide feedback.