search

ashawkey / stable-dreamfusion

Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.
Apache License 2.0
8k stars 710 forks source link

evaluation code request #205

Open jelleopard opened 1 year ago

jelleopard commented 1 year ago

HI, Could you provide the evaluation code and the test results with 153 prompts from the COCO validation subset? It seems the accuracy (50%）is much lower than R-precision mentioned on the paper.

zeng-yifei commented 1 year ago

Oh, this is not the offical implementation of the DreamFusion paper. Anyway, the base diffusion model is different.(Stable Diffusion v.s. Imagen) So I think there will not be a official evaluation code in this repo. And it's reasonable for the accuracy to be lower than the original paper.

BTW, if you don't count in the DMTet finetune in this repo, there is acutally no open-source code that could surpass the result in original dreamfusion at this moment lol. This repo is actually the best one now with DMTet.

jelleopard commented 1 year ago

i see. thanks for your response and your great work. But it seem the gap between our acurracy and the paper is too huge. i am not sure if it's our test code made this huge gap. i would like to know if you test the accuracy with R-precision and the results.

ashawkey commented 1 year ago

Sorry that I haven't tried that.

Leonever commented 8 months ago

I have the same question. And I can't figure out the main reason or the main part of the model caused the huge gap(about 30% R-precision, without DMTet finetune, in text case "a delicious hamburger.").