search

siliconflow / onediff

OneDiff: An out-of-the-box acceleration library for diffusion models.
https://github.com/siliconflow/onediff/wiki
Apache License 2.0
1.4k stars 85 forks source link

webui support controlnet extension #948

Closed marigoold closed 1 week ago

marigoold commented 2 weeks ago

only supported UNet inference with control, but controlnet model not supported now

lijunliangTG commented 1 week ago

分辨率在512*512的情况下 base模型为：AWPainting_v1.2.safetensors

Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 2.662 2.767s stock_mountain a_large_avalanchecannyFalse a_large_avalanchecannyTrue
Depth 2.535 2.545 s sk-b-dep a_large_avalancheDepthFalse a_large_avalancheDepthTrue
Segmentation 2.793 s 2.771 s sk-b-src a_large_avalancheSegmentationFalse a_large_avalancheSegmentationTrue
OpenPose 2.503 s 2.597 s an-pose a_large_avalancheopenposeTrue a_large_avalancheopenposeFalse
步骤 图片
先使用Canny生成图片，再同时用Canny+Openpose 生成图片，可以正常出图 image (3)
接着关闭Canny，然后仅开启Openpose 可以正常出图，可以正常出图 image (4)
接着关闭Openpose ，然后仅开启Canny可以正常出图，可以正常出图 image (5)
步骤 图片
同时用Depth+Openpose+ Canny 生成图片，可以正常出图 image (9)
关闭Canny，然后仅开启Depth+Openpose可以正常出图 image (8)
关闭Openpose ，然后仅开启Depth+ Canny可以正常出图，可以正常出图 image (7)
接着关闭Canny ，然后仅开启Canny可以正常出图，可以正常出图 image (10)
lijunliangTG commented 1 week ago

分辨率在1024*1024的情况下 base模型为：AWPainting_v1.2.safetensors

Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 17.575 s 20.900s stock_mountain a_large_avalanchecontrol_v11p_sd15_cannyTrue a_large_avalanchecontrol_v11p_sd15_cannyFalse
Depth 16.420 s 19.198 s sk-b-dep a_large_avalanchecontrol_v11f1p_sd15_depthFalse a_large_avalanchecontrol_v11f1p_sd15_depthTrue
Segmentation 16.560 s 21.455 s sk-b-src a_large_avalanchecontrol_sd15_segFalse a_large_avalanchecontrol_sd15_segTrue
OpenPose 16.484 s 21.781 s an-pose a_large_avalanchecontrol_sd15_openposeFalse a_large_avalanchecontrol_sd15_openposeTrue
lijunliangTG commented 1 week ago
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny 2.898 s 6.256s stock_mountain a_large_avalanchecontrol_v11p_sd15_cannyFalse a_large_avalanchecontrol_v11p_sd15_cannyTrue
Depth 2.645 s 6.198 s sk-b-dep a_large_avalanchecontrol_v11f1p_sd15_depthFalse a_large_avalanchecontrol_v11f1p_sd15_depthTrue
Segmentation 2.972 s 9.657 s sk-b-src a_large_avalanchecontrol_sd15_segFalse a_large_avalanchecontrol_sd15_segTrue
OpenPose 2.741 s 7.826 s an-pose a_large_avalanchecontrol_sd15_openposeFalse a_large_avalanchecontrol_sd15_openposeTrue
Control Type OneDiff 速度 原生速度 原图 output_img OneDiff_img
Canny X X
Depth X X
Segmentation X X
OpenPose X X
lijunliangTG commented 1 week ago

1、直接用sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdx不能正常运行，报错如sdxl_onediff_error.txt sdxl_onediff_error.txt

2、先使用sd_xl_base_1.0.safetensors+onediff，再sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdx，可以正常运行 sd_xl_base_1.0.safetensors+onediff sd_xl_base_1.0.safetensors+onediff+controlnet-canny-sdxl
image (14) image (15)