axodox / axodox-machinelearning

This repository contains a pure C++ ONNX implementation of multiple offline AI models, such as StableDiffusion (1.5 and XL), ControlNet, Midas, HED and OpenPose.
MIT License
605 stars 35 forks source link

[Feature Request] Support v-prediction #25

Open ZDisket opened 8 months ago

ZDisket commented 8 months ago

Some SD finetunes (and the official Stable Diffusion 2) use v-prediction because they observe that it's better Currently, v-prediction models will export with Olive and run fine with the ONNX diffusers pipeline, but not here. This is "a castle" on one I exported a castle - euler unmod

I tried modifying DpmPlusPlus2MScheduler::ApplyStep by modifying the predictedOriginalSample based off one of the diffusers implementations

        auto predictedOriginalSample = output.BinaryOperation<float>(input, [currentSigma](float a, float b) {
            float sigmaSquared = currentSigma * currentSigma;
            return (a * (-currentSigma / sqrt(sigmaSquared + 1))) + (b / sqrt(sigmaSquared + 1));
            });

I could only make the result semi-coherent

In DPMPlusPlus2MScheduler, the formula from HF's euler discrete works

        auto predictedOriginalSample = output.BinaryOperation<float>(input, [currentSigma](float model_output, float sample){
            float sigmaSquaredPlusOne = currentSigma * currentSigma + 1;
            return (model_output * (-currentSigma / std::sqrt(sigmaSquaredPlusOne))) + (sample / sigmaSquaredPlusOne);
            });

a castle - dpm fix

If you could finish this (haven't figured out EulerAncestralScheduler) (I have everything here), it would greatly improve compatibility. I uploaded the model in case it's useful.