search

siliconflow / onediff

OneDiff: An out-of-the-box acceleration library for diffusion models.
https://github.com/siliconflow/onediff/wiki
Apache License 2.0
1.4k stars 85 forks source link

How to use int8 for onediff Controlnet #979

Open cchen-reese opened 6 days ago

cchen-reese commented 6 days ago

I am using Onediff - Controlnet, load model of float16.

In your introduction, you used onediff int8, which is very effective in accelerating the model. I want to know if this is applicable to controlnet. If so, how can I use it?

CuddleSabe commented 6 days ago

only enterprise edition can use int8

cchen-reese commented 6 days ago

only enterprise edition can use int8

ok, thanks.

strint commented 6 days ago

int8 is not good to be compatible with Lora switching, and quantization is model related. It is not easy to use for most of the users.

If you have a lot of traffic on a single stable model, you are fit for int8. So we put it into the enterprise edition.

cchen-reese commented 6 days ago

int8 is not good to be compatible with Lora switching, and quantization is model related. It is not easy to use for most of the users.

If you have a lot of traffic on a single stable model, you are fit for int8. So we put it into the enterprise edition.

Thank you for your reply. By the way, do you see any degradation in the quality of image generation, under the usage of int8?

strint commented 6 days ago

int8 is not good to be compatible with Lora switching, and quantization is model related. It is not easy to use for most of the users. If you have a lot of traffic on a single stable model, you are fit for int8. So we put it into the enterprise edition.

Thank you for your reply. By the way, do you see any degradation in the quality of image generation, under the usage of int8?

https://github.com/siliconflow/onediff?tab=readme-ov-file#onediff-quality-evaluation