ImageDream Reconstruction

Peng Wang, Yichun Shi

Installation

This part is the same as original MVDream-threestudio. Skip it if you already have installed the environment.

Quickstart

Clone the modelcard on the Huggingface ImageDream Model Page under ./extern/ImageDream/release_models/

In the paper, we use the configuration with soft-shading. It would need an A100 GPU in most cases to compute normal:

export PYTHONPATH=$PYTHONPATH:./extern/ImageDream
image_file="./extern/ImageDream/assets/astronaut.png"
ckpt_file="./extern/ImageDream/release_models/ImageDream/sd-v2.1-base-4view-ipmv.pt"
cfg_file="./extern/ImageDream/imagedream/configs/sd_v2_base_ipmv.yaml"

python3 launch.py \
    --config configs/$method.yaml --train --gpu 0 \
    name="imagedream-sd21-shading" tag="astronaut" \
    system.prompt_processor.prompt="an astronaut riding a horse" \
    system.prompt_processor.image_path="${image_file}" \
    system.guidance.ckpt_path="${ckpt_file}" \
    system.guidance.config_path="${cfg_file}"

For diffusion only model, refer to subdir ./extern/ImageDream/ Check ./threestudio/scripts/run_imagedream.sh for a bash example.

Credits

This code is forked from threestudio and MVDream for SDS and 3D Generation.

Tips

Place the object in the center and do not make it too large/small in the image.
If you have an object cutting image edge, in config, tuning the parameters range of elevation and fov to be a larger range, e.g. [0, 30], otherwise, you may do image outpainting and follow tips 1.
Check the results with ImageDream diffusion model before using it in 3D rendering to save time.

PreComputed Results

Since there is some randomness in diffusion model and time costly to get baseline results. We put our pre-computed results for reproducing Tab.1 in the paper in a hugging face dataset card

Citing

If you find ImageDream helpful, please consider citing:

@article{wang2023imagedream,
  title={ImageDream: Image-Prompt Multi-view Diffusion for 3D Generation},
  author={Wang, Peng and Shi, Yichun},
  journal={arXiv preprint arXiv:2312.02201},
  year={2023}
}

bytedance / ImageDream

readme