search

ashawkey / stable-dreamfusion

Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.
Apache License 2.0
8k stars 710 forks source link

Latent nerf with magic 3d #138

Open Jainam2130 opened 1 year ago

Jainam2130 commented 1 year ago

Was anyone able to integrate the awsome magic3d improvements into latent nerf? For sd training in the latent space is extremely beneficial and am wondering how the normal pred from magic3d will affect it

I've tried to implement it over here but am getting shape mismatch errors and failing https://github.com/Jainam213/nerf

Please let me know if someone was able to implement it/could get my try, to work :)

If the quality improves enough we might not even need to backpropogate from latent space into rgb space shaving of 15 minutes

xuanyuzhang21 commented 1 year ago

Hi~ @Jainam213 Have you realized the stable dreamfusion with magic3d in latent space?

Jainam2130 commented 1 year ago

Hey! I sorry havent been able to iron out the errors yet but the major portion of it is done Maybe you can take a look? The shape mistmatch error is at

/src/latent_nerf/models/renderer.py line 355