ModelsLab / diffusers_plus_plus

Diffusers++: State-of-the-art diffusion models for image and audio generation in PyTorch

https://huggingface.co/docs/diffusers

Apache License 2.0

9 stars 2 forks source link

Add Hunyuan Diffusion Transformer #15

Closed shauray8 closed 4 days ago

shauray8 commented 1 month ago

What does this PR do?

Adds Hunyuan DiT which consists of adding a new DiT block with no Adaptive layer norm rather some neat integrations to the pipeline itself.

[ ] Add new DiT model from the paper
[ ] Integrate pipeline with the newer DiT block
[ ] Write tests
[ ] Write a training script wo/ the up casting is possible
[ ] migrate to sparse DiT blocks on training runs [EXP]
- [ ] SDXL VAE is too 'compressed' which makes it harder to finetune maybe transfer to SD1/2 latent space while training
  
  The model seems to be a little under trained: the architecture on the other hand looks really good

Fixes #12