Diffusion Image Analogies

This is the official repository for the Diffusion Image Analogies paper published at the SIGGRAPH 2023 Conference Proceedings.

Installation

1. Clone the repo

   git clone --recurse-submodules https://github.com/subrtadel/DIA.git
   cd ./DIA

2. Create environment

   conda create -n dia_env
   conda activate dia_env
   conda install python=3.8.5 pip=20.3 cudatoolkit=11.3 pytorch=1.11.0 torchvision=0.12.0 numpy=1.19.2 -c pytorch -c nvidia -c conda-forge -c defaults

3. Install packages

   pip install -r requirements.txt
   cd ./stable-diffusion/
   pip install -e git+https://github.com/CompVis/taming-transformers.git@master#egg=taming-transformers
   pip install -e .

4. Download the sd-v1-4.ckpt model and put it into correct folder

   mkdir -p ./models/ldm/stable-diffusion-v1/
   

5. Install Image Magick.

(back to top)

Usage

1. Upload images into ./dataset/raw_data/ folder.
   

2. Run process_new_data.py. The images are assigned file_ids in a %05d format.
   

3. Define the triplets in a .csv file. Refer to the images by their file_id. Example file is triplets.csv. First column specifies the A input, second the A' and the third B input. Either with of without filename suffixes is fine.
   

4. Run the precompute_noises_and_conditionings.py script. This may take a while.

   python precompute_noises_and_conditionings.py \
       --config ./config/parameter_estimation.yaml \
       --inversion_subfolder noise \
       --token_subfolder tokens \ 
       --triplet_file triplets.csv \
       --data_path ./dataset/data/

   
   

5. Check the ./config/analogy_params.yaml.
   

6. Run the do_analogies.py script.

   python do_analogies.py \
       --config ./config/parameter_estimation.yaml \
       --inversion_subfolder noise \
       --token_subfolder tokens \ 
       --output_subfolder analogies \
       --triplet_file triplets.csv \
       --data_path ./dataset/data/

BibTeX

@inproceedings{Subrtova2023DIA,
    title = {Diffusion Image Analogies},
    author = {A. \v{S}ubrtov\'{a} and M. Luk\'{a}\v{c} and J. \v{C}ech and D. Futschik and E. Shechtman  and D. S\'{y}kora},
    booktitle = {ACM SIGGRAPH 2023 Conference Proceedings},
    year = {2023}
  }