vae_decoder_qu8/range_data.txt value generation

vitoplantamura / OnnxStream

Lightweight inference library for ONNX files, written in C++. It can run Stable Diffusion XL 1.0 on a RPI Zero 2 (or in 298MB of RAM) but also Mistral 7B on desktops and servers. ARM, x86, WASM, RISC-V supported. Accelerated by XNNPACK.

https://yolo.vitoplantamura.com/

Other

1.86k stars 84 forks source link

vae_decoder_qu8/range_data.txt value generation #63

Closed thexa4 closed 7 months ago

thexa4 commented 7 months ago

I'm trying to convert a checkpoint to run with OnnxStream but I can't seem to figure out how to generate the range_data.txt file.

Are there default values that you should fill in before running with --decoder-calibrate? If the file doesn't exist the application quits before running the calibration.

vitoplantamura commented 7 months ago

hi,

an empty file should be sufficient for the first run.

However, consider that you probably don't need to calibrate the VAE decoder, for these reasons:

1) typically the SD 1.5 VAE decoder is common to all SD 1.5 derivations. Typically finetune only alters the UNET model.

2) the quantized version of the VAE decoder is only needed to keep RAM usage under 300MB, and is only used when the "--rpi-lowmem" option is specified.

Vito