search

ethanweber / nerfiller

NeRFiller project https://ethanweber.me/nerfiller/
Apache License 2.0
101 stars 3 forks source link

SDS + Gridding #15

Closed jingyushi closed 2 months ago

jingyushi commented 2 months ago

Hi Ethan,

Thank you so much for the interesting work. I am currently working on a project utilizing SDS for NeRF optimization. In your implementation, Grid Prior seems like a promising trial for me. I saw you commented in your code that Gridding is not working for SDS right now (here). Could you please elaborate more on this? Much appreciate!

ethanweber commented 2 months ago

Hey! So that comment is misleading and SDS should work too! I just didn't implement it since I prefer using dataset updates instead of SDS.

In summary, I only experimented with SDS vs. dataset updates (in an apples-to-apples comparison) for full images. This is not to say that SDS with grid prior cannot be used, though! Feel free to implement it for SDS if you'd like. :)

However! One issue you'll find with SDS is that it will be challenging to use > 4 images for joint inpainting. This requires averaging over multiple grid permutations while denoising images, and I don't see a straightforward way to do this with SDS. Hopefully this helps! Thanks for your patience on my reply.