search

ashawkey / stable-dreamfusion

Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.
Apache License 2.0
8.2k stars 720 forks source link

New Paper: ProlificDreamer - several ideas to improve detail/quality #296

Open mhale1 opened 1 year ago

mhale1 commented 1 year ago

Maybe you've already seen this, but a new paper just came out. They call it VDS optimization instead of SDS. Fixes saturated colors, lack of variety, and smoothed details. Requires an extra LoRA. But even without Zero123 they are getting some very good results bumping up to 512 resolution. Check out their pineapple. https://ml.cs.tsinghua.edu.cn/prolificdreamer/

https://github.com/ashawkey/stable-dreamfusion/assets/5043964/af385718-5509-4f87-86f7-9ff8d57f0f4d

pkulwj1994 commented 1 year ago

This works seems very exciting, is there a plan to incorporate the techinques into Stable-Dreamfusion projects?

jclarkk commented 1 year ago

https://arxiv.org/abs/2305.18766

And now another loss function comes to light :)

Seems very interesting and it's able to skip the refinement stage.

mdarhdarz commented 1 year ago

https://arxiv.org/abs/2305.18766

And now another loss function comes to light :)

Seems very interesting and it's able to skip the refinement stage.

I implement BGT+ loss and test it on the dmtet stage, but it did not show improvement. If kiui has interest to implement it, I would like to compare the code and know any details are ignored or misunderstood