search

nerfstudio-project / nerfstudio

A collaboration friendly studio for NeRFs
https://docs.nerf.studio
Apache License 2.0
9.58k stars 1.31k forks source link

Integrate SAM model with Nerfstudio #3248

Open abrahamezzeddine opened 5 months ago

abrahamezzeddine commented 5 months ago

Hello,

I am wondering if it’s possible to integrate SAM masks for each frame with splatfacto, so that each point perhaps also gets an additional masking semantic attribute based on mask?

Any suggestion would be very helpful.

Thank you.

TamirCohen commented 5 months ago

I think it might be cool to add implementation of https://feature-3dgs.github.io/ (maybe as an external method). Then you can add SAM and any other feature field easly.

I am not nerfstudio developer / maintainer, I just think it is a useful feature. I can help implement it.

abrahamezzeddine commented 5 months ago

I think it might be cool to add implementation of https://feature-3dgs.github.io/ (maybe as an external method). Then you can add SAM and any other feature field easly.

I am not nerfstudio developer / maintainer, I just think it is a useful feature. I can help implement it.

Would be great! I would be happy to test it out and provide feedback.