threestudio-project / threestudio

A unified framework for 3D content generation.
Apache License 2.0
6.14k stars 471 forks source link

Add TP-SDS (and potentially VSD) to improve quality/diversity #194

Open yankeesong opened 1 year ago

yankeesong commented 1 year ago

TP-SDS This paper shows that using monotonically non-increasing time-sampling in diffusion models shows better results than uniform sampling. I can try to implement it.

thuliu-yt16 commented 1 year ago

Of course, PR is welcome!

yuanzhi-zhu commented 1 year ago

@yankeesong I tried it with 2d experiments here https://github.com/yuanzhi-zhu/prolific_dreamer2d and found that Dreamtime only helps sds a bit but not vsd.

Would like to see your implementation!!

yankeesong commented 1 year ago

@yuanzhi-zhu I suspect this is because for SDS the diffusion is frozen whereas for VSD a LoRA is trained. If one trains the LoRA with TP (i.e. with non-uniform time sampling), they might introduce inductive bias.