search

ironjr / StreamMultiDiffusion

Official code for the paper "StreamMultiDiffusion: Real-Time Interactive Generation with Region-Based Semantic Control."
https://jaerinlee.com/research/streammultidiffusion
MIT License
518 stars 43 forks source link

checkpoint models #4

Closed redclif43 closed 5 months ago

redclif43 commented 5 months ago

How do I change or add checkpoints?

ironjr commented 5 months ago

If you are using semantic palette demo at src/app/semantic_palette/app.py or streaming demo at src/app/stream/app.py you can specify options for the model checkpoint in the format --model "KBlueLeaf/kohaku-v2.1".

And... I just have added .safetensors support for both of the demo app. The instructions are quite easy.

  1. Save <your model>.safetensors or a symbolic link to the actual file to demo/semantic_palette/checkpoints or demo/stream/checkpoints.
  2. Load your model with python app.py --model <your model>.safetensors.

I will add the instructions in the readme as well.

Please note that we are currently supporting only SD1.5, but SDXL is on the way.

redclif43 commented 5 months ago

but is it possible to use realistic models?

ironjr commented 5 months ago

Sure, why not? :)

git clone https://github.com/ironjr/StreamMultiDiffusion
cd demo/semantic_palette_sdxl # or demo/semantic_palette if you want SD1.5 checkpoints

mv "<path_to_your_model.safetensors>" checkpoints/

pip install -r requirements.txt
python app.py --model "<your_model_name_relative_path_from_checkpoints_directory>.safetensors"
# Then, open your browser and type: https://localhost:8000

https://github.com/ironjr/StreamMultiDiffusion/assets/12259041/9a6bb02b-7dca-4dd0-a1bc-6153dde1571d