search

NIRVANALAN / LN3Diff

[ECCV-2024] LN3Diff creates high-quality 3D object mesh from text within 8 V100-SECONDS.
https://nirvanalan.github.io/projects/ln3diff/
Other
155 stars 10 forks source link

Reproduction issue #9

Open stalkerrush opened 1 week ago

stalkerrush commented 1 week ago

Hi @NIRVANALAN , thanks for releasing your work! I am trying to compare our ongoing work to yours, but I am facing some issues of reproduction. Most likely it is something I didn't do right, but I am getting weird results based on your instructions. So the first thing I tried is to run the local gradio demo, after downloading both your "checkpoints/objaverse/model_rec1890000.pt" and "checkpoints/objaverse/objaverse-dit/i23d/model_joint_denoise_rec_model2990000.pt" and placing it into the right place. The result I get is something like below: image Then I thought maybe the checkpoint has some issue, so I went to your huggingface spaces files and directly downloaded your app.py and the i23d_args.json and run it locally (I believe this would load from your huggingface instead of local ckpts). However, the results are still the same. Do you know what could go wrong here?

Thank you.

NIRVANALAN commented 1 week ago

Hi, thanks for your interest in your work. The result is wired and is likely to be due to the checkpoint is not loaded correctly.

First, could you check the online gradio demo, to see whether our checkpoint can yield reasonable result?

Besides, when running the gradio demo locally, could you check the log and see whether there is some log like "xxx .ckpt" is loaded? There may be some typo in the $resume_checkpoint of bash file, and I will test again asap.

Please keep me updated with your progress.

NIRVANALAN commented 1 week ago

Hi, I double checked the result on both the online gradio demo and local inference pipeline, and it should be right. Could you check whether the checkpoint is loaded right (be aware of the corresponding logs), and if it still fails, perhaps some of the environment packages are wrong. You can keep me updated in this case, and use the online gradio demo for your comparison (you can download the output textured mesh from the gradio demo).

NIRVANALAN commented 1 week ago

Another solution is cloning the repo from Hugging face demo and run python app.py locally. It should directly load the checkpoint from huggingface hub.

stalkerrush commented 4 days ago

Thanks for your reply. I actually have tried both before opening the issue, but I am getting consistent results. I also double checked that the checkpoint was loaded. Yeah, environment might be one possibility. I can try this on a different machine some time later.

NIRVANALAN commented 3 days ago

Hi! Thanks for your reply. That situation is wired, and please keep me updated with your progress. Besides, doe the online gradio demo works well? If so, that may be due to the env issue. I will try to reproduce the results on another server soon later, let's keep in touch.

NIRVANALAN commented 13 hours ago

Hi @stalkerrush , I have double checked the performance on another server, and it looks fine. Now I just updated the repo to support direct (by default) ckpt loading from hugging face, and you can directly launch the gradio demo locally according to the instructions. Besides, I have removed the apex dependency in the code, and all modules shall be supported by xformers and torch. Please let me know if you still have the reproduction issue, and I am ready to help.