SchedulerBase oversight and others.

[follyfoxe]

I'll be honest. Some of the code here is... not very good in my opinion. At least for what I consider acceptable C# code personally. There's a lot of issues with this. Cloning this repo and seeing the source files, it was quite painful; Static classes instead of objects that implement IDisposable or any way of cleaning up... ToArray() being overused everywhere... SchedulerBase and LMSDiscreteScheduler having conflicting variable names regarding inheritance... using the IEnumarable<T>.Count() extension method when Array.Length or List<T>.Count is available... Among other things.

Some of the code here.... honestly looks auto-generated. And I want to report, a really really small thing. But it messes up running the model with larger StableDiffusionConfig.NumInferenceSteps (tried with 30) This line of code here, in SchedulerBase.cs

// If timesteps[i] is greater than the last element in range, use the last value in sigmas
else if (index == -range.Length - 1)
{
       result[i] = sigmas[-1];
}

It may look completely fine to a python delevoper. But sadly, things aren't like that in C# A correct way would be: result[i] = sigmas[sigmas.Count - 1]; Otherwise, at larger NumInferenceSteps, it crashes due to the index being out of range. Not sure if this oversight is in other places. I haven't examined all of the classes.

Don't get me wrong, I'm really glad this repository exists at all, otherwise, I probably wouldn't have figured out how to run this thing in C# Hope this helps!!!

Edit: NamedOnnxValue.CreateFromTensor("timestep", new DenseTensor<Float16>(new Float16[] { BitConverter.HalfToUInt16Bits((Half)timeStep) }, new int[] { 1 })) In my case, the fp16 model uses int64 for the "timestamp" input. Therefore this line in the DirectML fp16 branch, UNet.cs had to be changed for this specific case, not sure if it applies to anything else. NamedOnnxValue.CreateFromTensor("timestep", new DenseTensor<long>(new long[] { timeStep }, new int[] { 1 }))

[yakovw]

I'm glad that something at all works for you, because most of those who tried it before you, just didn't work for them. Those who have already worked for him, worked only on a processor, not on a GPU It is indeed disappointing that there is no normal C# library that supports running StableDiffusion I'm glad that something at all works for you, because most of those who tried it before you, just didn't work for them. Those who have already worked for him, worked only on a processor, not on a GPU It is indeed disappointing that there is no normal C# library that supports running StableDiffusion

[follyfoxe]

I have only gotten it to work on the CPU so far. I'm currently trying to make it run with the GPU. I'm using a custom model converted to onnx. I've tried running it with CUDA but got an error regarding the inability to allocate some buffer. I'm assuming it was because of the big model size and my extremely low dedicated vram; So with that in mind, I converted the model to use fp16 and lowered the size significantly. Really hoping it'll work. I'll keep you updated!

Update: I've got DirectML sort of working! I'll take that as a win.

[yakovw]

OK happy for you I'm sorry mine doesn't work I am waiting for a project similar to https://github.com/SciSharp/LLamaSharp

I have only gotten it to work on the CPU so far. I'm currently trying to make it run with the GPU. I'm using a custom model converted to onnx. I've tried running it with CUDA but got an error regarding the inability to allocate some buffer. I'm assuming it was because of the big model size and my extremely low dedicated vram; So with that in mind, I converted the model to use fp16 and lowered the size significantly. Really hoping it'll work. I'll keep you updated!

Update: I've got DirectML sort of working! I'll take that as a win.

[follyfoxe]

OK happy for you I'm sorry mine doesn't work I am waiting for a project similar to https://github.com/SciSharp/LLamaSharp

I'm not exactly sure what you mean? In the case that you are one of the creators of this project: Don't worry! It's working now. Otherwise, I'm sorry you haven't been able to get it to work.

Stable diffusion models are often in .ckpt or .safetensors format. The way I managed to run it with DirectML was by converting a .safetensors sd model to diffusers with a python script, then converting that to onnx with optimum-cli, and finally running another separate python script for optimizing sd onnx models. It took so long to figure out. There aren't too many resources out there sadly...

Hopefully with time, more and more libraries for C# and AI will start appearing. And the process won't be as cumbersome as it is currently.

[yakovw]

OK happy for you I'm sorry mine doesn't work I am waiting for a project similar to https://github.com/SciSharp/LLamaSharp

I'm not exactly sure what you mean? In the case that you are one of the creators of this project: Don't worry! It's working now. Otherwise, I'm sorry you haven't been able to get it to work.

Stable diffusion models are often in .ckpt or .safetensors format. The way I managed to run it with DirectML was by converting a .safetensors sd model to diffusers with a python script, then converting that to onnx with optimum-cli, and finally running another separate python script for optimizing sd onnx models. It took so long to figure out. There aren't too many resources out there sadly...

Hopefully with time, more and more libraries for C# and AI will start appearing. And the process won't be as cumbersome as it is currently.

I did mean that it is not possible to run a safetensors file and it must be converted first.

But if it at least works perfectly, it will already be progress

I will try again later

Thank you

[LeonNerd]

hi,I would like to ask, how do I run the fp16 model in DirectML, the branch I use is StableDiffusion-direct-ML-EP. My FP16 model is through --python optimize_pipeline.py -i ./sd-v1-5 -o ./sd-v1-5-fp16 --float16 in https://github.com/microsoft/onnxruntime/tree/main/onnxruntime/python/tools/transformers/models/stable_diffusion. what should i do?

[cassiebreviu]

@LeonNerd You might want to try this version of the code for fp16 directml sample on the onnx runtime org here: https://github.com/onnxruntime/StableDiffusion-v1.5-Onnx-Demo/tree/main