search

thu-nics / ViDiT-Q

ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation
https://a-suozhang.xyz/viditq.github.io/
34 stars 4 forks source link

cond and uncond quantization issue #4

Open Andy0422 opened 4 months ago

Andy0422 commented 4 months ago

I just confused that i cannot get the clue about the cond and uncond quantization part you claim in your paper in the code, could you give me a hand?

A-suozhang commented 4 months ago

The classifier-free guidance process consists of two network inference, once with the control signal, and once without, the results are then weighted sum together. These two inference have activation variations, and thus should use different quantization parameters.

Normally, the two inference is conducted in parallel in the batch dimension, and the quantizer automatically calculated the activation scaling factor for each token (of all batches). To calculate the quantization parameter respectively for cond and uncond, we need to conduct two separate inference.

The CFG split code is presented here (apology for it's hidden inside the scheduler), by turning the CFG_SPLIT=True, you could conduct 2 separate inference, and save the quantization parameters for the cond and uncond inference respectively.

https://github.com/A-suozhang/ViDiT-Q/blob/fb2c8486a63aee5071a80d7382802ace0b08aeb7/t2v/opensora/schedulers/iddpm/__init__.py#L140