OneDiff

OneDiff is an out-of-the-box acceleration library for diffusion models, it provides:

• PyTorch Module compilation tools and strong optimized GPU Kernels for diffusion models
• Out-of-the-box acceleration for popular UIs/libs
  • OneDiff for HF diffusers 🤗
  • OneDiff for ComfyUI
  • OneDiff for Stable Diffusion web UI

For example:

OneDiff is the abbreviation of "one line of code to accelerate diffusion models".

News

The latest news:

• :rocket:OneDiff 1.0 is out! (Acceleration of SD & SVD with one line of code)
• :rocket:Accelerating Stable Video Diffusion 3x faster with OneDiff DeepCache + Int8
• :rocket:Accelerating SDXL 3x faster with DeepCache and OneDiff

Community and Support

Here is the introduction of OneDiff Community.

• Create an issue
• Chat in Discord:
• Email for Enterprise Edition or other business inquiries: contact@siliconflow.com

OS and GPU Compatibility

• Linux

  • If you want to use OneDiff on Windows, please use it under WSL.
  • The guide to install OneDiff in WSL2.

• NVIDIA GPUs

  • Compatibility with Nvidia GPUs.

The Full Introduction of OneDiff:

• OneDiff
• News
• Community and Support
• OS and GPU Compatibility
• About OneDiff
  • Architecture
  • State-of-the-art performance
  • SDXL E2E time
  • SVD E2E time
  • Features
  • Acceleration for State-of-the-art models
  • Acceleration for production environment
  • PyTorch Module compilation
  • Avoid compilation time for new input shape
  • Avoid compilation time for online serving
  • OneDiff Quality Evaluation
  • OneDiff Enterprise Edition
• Installation
  • OneDiff Installation
  • Install a compiler backend
    • (Optional) Install Nexfort
    • (Optional) Install OneFlow
  • 2. Install torch and diffusers
  • 3. Install OneDiff
  • 4. (Optional)Login huggingface-cli

About OneDiff

Architecture

OneDiff interfaces with various front-end sd frameworks upward, and uses a custom virtual machine mixed with PyTorch as the inference engine downward.

State-of-the-art performance

SDXL E2E time

• Model stabilityai/stable-diffusion-xl-base-1.0;
• Image size 1024*1024, batch size 1, steps 30;
• NVIDIA A100 80G SXM4;

SVD E2E time

• Model stabilityai/stable-video-diffusion-img2vid-xt;
• Image size 576*1024, batch size 1, steps 25, decoder chunk size 5;
• NVIDIA A100 80G SXM4;

Note that we haven't got the way to run SVD with TensorRT on Feb 29 2024.

Features

Acceleration for State-of-the-art models

OneDiff supports the acceleration for SOTA models.

• stable: release for public usage, and has long-term support;
• beta: release for professional usage, and has long-term support;
• alpha: early release for expert usage, and should be careful to use;

Note: Enterprise Edition contains all the functionality in Community Edition.

Acceleration for production environment

PyTorch Module compilation

• compilation with oneflow_compile

  Avoid compilation time for new input shape

• Support Multi-resolution input

  Avoid compilation time for online serving

  Compile and save the compiled result offline, then load it online for serving

• Save and Load the compiled graph
• Change device of the compiled graph to do multi-process serving
  • Compile at one device(such as device 0), then use the compiled result to other device(such as device 1~7).

OneDiff Quality Evaluation

We also maintain a repository for benchmarking the quality of generation after acceleration using OneDiff: OneDiffGenMetrics

OneDiff Enterprise Edition

If you need Enterprise-level Support for your system or business, you can

• Subscribe to the OneDiff Enterprise Edition directly through our website. Upon purchase, you'll gain immediate access to comprehensive support: https://siliconflow.com/onediff.html
• For a more personalized approach, please email us at contact@siliconflow.com. Include details about your use case, deployment size, and any specific needs you might have.

The OneDiff Enterprise Edition is available for a monthly subscription and is designed to be cost-effective, even for systems utilizing a single GPU.

Installation

OneDiff Installation

Install a compiler backend

When considering the choice between OneFlow and Nexfort, either one is optional, and only one is needed.

• For DiT structural models or H100 devices, it is recommended to use Nexfort.

• For all other cases, it is recommended to use OneFlow. Note that optimizations within OneFlow will gradually transition to Nexfort in the future.

(Optional) Install Nexfort

The detailed introduction of Nexfort is here.

python3 -m  pip install -U torch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 torchao==0.1
python3 -m  pip install -U nexfort

(Optional) Install OneFlow

NOTE: We have updated OneFlow frequently for OneDiff, so please install OneFlow by the links below.

• CUDA 11.8

  For NA/EU users

  python3 -m pip install -U --pre oneflow -f https://github.com/siliconflow/oneflow_releases/releases/expanded_assets/community_cu118

  For CN users

  python3 -m pip install -U --pre oneflow -f https://oneflow-pro.oss-cn-beijing.aliyuncs.com/branch/community/cu118

2. Install torch and diffusers

Note: You can choose the latest versions you want for diffusers or transformers.

python3 -m pip install "torch" "transformers==4.27.1" "diffusers[torch]==0.19.3"

3. Install OneDiff

• From PyPI

  python3 -m pip install --pre onediff

• From source

  git clone https://github.com/siliconflow/onediff.git
  cd onediff && python3 -m pip install -e .

NOTE: If you intend to utilize plugins for ComfyUI/StableDiffusion-WebUI, we highly recommend installing OneDiff from the source rather than PyPI. This is necessary as you'll need to manually copy (or create a soft link) for the relevant code into the extension folder of these UIs/Libs.

4. (Optional)Login huggingface-cli

python3 -m pip install huggingface_hub
huggingface-cli login