search

PixArt-alpha / PixArt-sigma

PixArt-Σ: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation
https://pixart-alpha.github.io/PixArt-sigma-project/
GNU Affero General Public License v3.0
1.44k stars 68 forks source link

Is it possible to use Controlnet, IPAdapter, and Lora #55

Open libai-lab opened 2 months ago

libai-lab commented 2 months ago

Is it possible to use Controlnet, IPAdapter, and Lora

lawrence-cj commented 2 months ago

We have trained ControlNet and Lora for PixArt-Alpha. If you want a use them here, need to train a new one.

xzwthu commented 2 months ago

I used the ControlNet code in PixArt-Alpha but got an error: No such file or directory: 'output/pretrained_models/ControlNetHED.pth' Has the pre-trained model been released? How can I get this file?

lawrence-cj commented 2 months ago

https://huggingface.co/PixArt-alpha/PixArt-alpha/blob/main/ControlNetHED.pth

dangweili commented 2 months ago

Is any plane to train and release ControlNet models, such as canny, hed, depth etc. in the near future?

lawrence-cj commented 2 months ago

Nope. Will only release the training script.

universewill commented 2 months ago

Nope. Will only release the training script.

When will the training script of controlnet released? How much vram needed to train controlnet of this sigma model?

lawrence-cj commented 2 months ago

Lora released.