adding PAG Support for Hunyuan-DIT and Pixart-Sigma

[yiyixuxu]

we recently added PAG support for SDXL. Is Anyone interested in extending PAG support to Hunyuan-DIT and Pixart-Sigma? There is no implementation available, so it is a bit of a research-oriented project (= fun!!). and you can get directly feedbacks from the authors @sunovivid @HyoungwonCho

to add PAG support to new models:

• I think you should be able to use PAGMixin as it is (or with some modification)(https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/pag/pag_utils.py#L27)
• you will need to make PAG attention processors for the new model https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py#L2564 based on the attention processor that the model uses, e.g. for Hunyuan-DIT, you need to make a HunyuanPAGIdentitySelfAttnProcessor2_0 and HunyuanPAGCFGIdentitySelfAttnProcessor2_0 based on HunyuanAttnProcessor2_0 https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/attention_processor.py#L1499
• you will need to make a HunyuanPAGPipeline /PixartSigmaPAGPipeline under the pag folder (for now!)

[yiyixuxu]

cc @a-r-r-o-w @sayakpaul in case you're interested!

[a-r-r-o-w]

I'd love to work on this, thanks! Taking up Hunyuan :)

[sayakpaul]

Will take PixArt-Sigma. Let's jam 🎸