search

kamalkraj / stable-diffusion-tritonserver

Deploy stable diffusion model with onnx/tenorrt + tritonserver
Apache License 2.0
122 stars 19 forks source link

Model exports for Tritonserver #1

Closed iborko closed 2 years ago

iborko commented 2 years ago

Hi! First of all, great repo and very useful. I've been using it with your model export (from huggingface) and everything works great.

But now I have to deploy a different version of Stable Diffusion with Triton server. I saw that you mentioned that you used this script: https://github.com/harishanand95/diffusers/blob/dml/examples/inference/save_onnx.py for model export. That script works for fp32, but how you exported it for fp16? Have you used some kind of converter? Do you have an example of fp16 export?

Thanks :)

kamalkraj commented 2 years ago

Hi @iborko,

import onnx
import onnxconverter_common

new_onnx_model = onnxconverter_common.convert_float_to_float16_model_path("stable-diffusion-v1-4-onnx/unet/1/unet.onnx",keep_io_types=True)
onnx.save(new_onnx_model, 'stable-diffusion-v1-4-onnx/unet/1/unet_fp16.onnx')
iborko commented 2 years ago

Thank you very much!

lolagiscard commented 1 year ago

Hi @iborko

Can you give some details about your hardware and the versions of diffusers and transformers you have been testing to make this Repo running ?

Thanks

iborko commented 1 year ago

I tested this repository on both Amazon G4dn and G5 instances.

You can find exact package versions in my fork: https://github.com/velebit-ai/stable-diffusion-tritonserver

Version from this repo doesn't work anymore because many packages changed their latest versions (including pytorch, diffusers, ...)

kamalkraj commented 1 year ago

Please checkout v2

let me know any issues