Suggestions, new ideas and general talk

[Amin456789]

Hello! thank u for this! i'm a huge fan of onnx as it is very cpu friendly. could u please try and make AnimateDiff to work with onnx too? it will be great to have text 2 gif and image 2 gif with AnimateDiff for onnx cpu in the future

kind regards

[Amin456789]

@saddam213 i see u r working on a gui! thank u for all ur efforts man!

i have some good news, there is a new sampler named LCM which has a very fast speed time and much much less steps are needed from now on! just with 3-4 steps u have a perfect image! it has a specific model which has to be converted to onnx i think, but the model is very very good. it will be great if we can have this sampler in ur gui in the future so we can be free from 20-30 or more steps! please search = latent-consistency-model by luosiallen on github for more info, it is very fresh and new the model is 3.4gb, i think it is f32, maybe we can have a f16 onnx of it too?

[Amin456789]

i see u r so active these days Adam! GO GO GO!

[Amin456789]

Hey guys Adam, if u ever wanted to bring LCM to onnx here is a repo that will help u: https://github.com/huggingface/optimum/pull/1469

and getting help of how bringing it to cpu and img2img etc... [not onnx but can get the idea]: https://github.com/0xbitches/sd-webui-lcm

and for getting help from implementing text2gif img2gif and knowing about frames and stuff this webui extension can help [not onnx, just the idea]: https://github.com/continue-revolution/sd-webui-animatediff

[saddam213]

Hopefully someone will convert some models and I can give LCM a shot

Unfortunately my lack of Python knowledge is a blocker for stuff like this :(

One of the main reasons I started OnnxStack is because I dont know Python.

[Amin456789]

hmm, in that repo people i think were generating images, maybe we can convert it with that onnx conversation? anyway, made a topic in optimum repo so we can get an answer: https://github.com/huggingface/optimum/issues/1497

ps: it is never too late to learn new stuff, i think u it will be great if learn python too, it will definitely will help u in ur career in the future. but hey, i remember i wanted to learn programming when i was a teenager and bought a Delphi language CD tutorial, after 5 minutes i dropped the disc away :d programming wasn't my thing, but for u it is a bless, so play with python sometimes... who knows buddy

[JohnClaw]

Hopefully someone will convert some models and I can give LCM a shot

And what about OpenVino? Could you implement it in OnnxStack, please?

[Amin456789]

from things i heard onnx and openvino almost have the same speed in cpu, so having one of them will be ok i guess. i can't wait for animatediff and lcm come to onnxstack to have everything in one place, my AI apps folder is a mess

[Amin456789]

Adam, u can convert the LCM model easily, i got an answer, please convert and test it out it is easy just like converting any other models, scripts and tutorial are in the answer here; https://github.com/huggingface/optimum/issues/1497

[Amin456789]

btw Adam, it will be great if u share the model in huggingface when you converted it after the test, please test it with fp16 version too if possible. when everything is good contact LCM official repo, they will put ur repo on the list which will be a boost for ur page too!

I am very grateful for whatever u do for us, sometimes my requests are a lot i know, but hey, i am so excited about this project ;) A BIG THANK U

[saddam213]

Adam, u can convert the LCM model easily, i got an answer, please convert and test it out it is easy just like converting any other models, scripts and tutorial are in the answer here; huggingface/optimum#1497

I tried for about 5 hours last night, however could not even install optimum-cl, just failed with an error, I gave up

Python just sucks on windows, nothing ever works for me, Im sure someone will make a model Onnx soon

[saddam213]

Hopefully someone will convert some models and I can give LCM a shot

And what about OpenVino? Could you implement it in OnnxStack, please?

That is something I might look at after OnnxStack, but its not a good fit for this project as this is Onnx integrations only

[Amin456789]

hope so, other than LCM, i really love this model images, they are so beautiful. couldn't find something close to it yet.

[saddam213]

I will be starting LCM this week as I have hit a roadblock with AnimatedDiff, I simply dont have the GPU power to test and keep my sanity, I only have a 3090, and it takes sooo long to generate a vid that its unbearable to debug :/

There is only so many times a man can wait 10min for a 24 frame vid thats all noise, lol

I have plenty of work to do before I need an LCM Onnx model, I still need to make a c# version of the LCMScheduler and diffusion loop, hopefully by the time I have finished that someone has figured out the coverting part :p

[saddam213]

Good News, we have a dev on the team @TheyCallMeHex he has this all sorted and is now converting models to onnx as we speak

The following are other stable diffusion models he has converted to ONNX and tested with OnnxStack.

• OpenJourney V4
• DreamLike PhotoReal 2.0
• CyberPunk Anime Diffusion
• InkPunk Diffusion
• Mo-Di Diffusion
• epiCRealism
• Comic Diffusion
• Redshift Diffusion

He is now looking at the LCM model :)

[Amin456789]

oh goodie, cant wait for the lcm! and about the animatediff, well maybe u should lower the settings way below like 8 frame to test...

[TheyCallMeHex]

I've converted the LCM-Dreamshaper-V7 model to ONNX, but wont be able to test it with OnnxStack until @saddam213 finished the C# scheduler for LCM.

https://huggingface.co/TheyCallMeHex/LCM-Dreamshaper-V7-ONNX

So as of now, this is completely untested(Other than it 100% wont work in OnnxStack yet)

[Amin456789]

great thanks!

[Amin456789]

now i have an idea Adam, if u managed to make the LCM to work after that u can connect LCM to Animatediff somehow, there is a repo for comfyui that u can connect lcm to animatediff now in my test LCM only needs 3 steps, not even 4. so i think less steps will make the process for u easier to work on animatediff also, u can use fp16 models for animatediff, they are in the description [model zoo] of continue-revolution repo, less size maybe faster generation, u should play with other settings such as context batch size [lower them] to see if it will be faster or not

[saddam213]

now i have an idea Adam, if u managed to make the LCM to work after that u can connect LCM to Animatediff somehow, there is a repo for comfyui that u can connect lcm to animatediff now in my test LCM only needs 3 steps, not even 4. so i think less steps will make the process for u easier to work on animatediff also, u can use fp16 models for animatediff, they are in the description [model zoo] of continue-revolution repo, less size maybe faster generation, u should play with other settings such as context batch size [lower them] to see if it will be faster or not

If I was modifying a python app then yes I could just change its settings, however in my case its not quite that easy, I don't have any settings to change like batch size, as I have not written them yet, I cant use any of that python code, I can use it as a reference but all implementation needs to be done from scratch, and its slow, I really need more GPU power, its just what it is

[saddam213]

But LCM uses less resources, so good place to start until I get a better GPU.

Been looking for an excuse to get an A100 80GB, maybe now is the time :)

[Amin456789]

hmm, i never tried any of this online ones, but i think u can rent very good gpus for 1 hour very cheaply in vast.ai site A40 GPU for 1 hour = $0.40

[saddam213]

TBH its just not that important to me, sorry, but I would rather just take my time

[saddam213]

Just got LCM TextToImage working!!!!!

Backend only, but I will update the UIs in the morning

Image generated with 6 steps 1.5seconds

All that math hurt my brain and need to have a sleep, LOL

[saddam213]

LCM ImageToImage now implemented

[JohnClaw]

Just got LCM TextToImage working!!!!!

Great! Thank you very much. Your app may become the best but only with OpenVino. FastSDCpu (https://github.com/rupeshs/fastsdcpu) combines LCM and OpenVino and generates images very fast. But it is written in Python. Python is slower than C#. So if you combine LCM and OpenVino in OnnxStack, your app will be faster than FastSDCpu.

[saddam213]

They can do 4 steps in 10 seconds, OnnxStack takes 12, pretty sure I can optimize it a bit more and get it close to 10, maybe below

But to convert the whole project now to OpenVino for a few seconds seems overkill

[saddam213]

I have opened a discussion section for new ideas etc :)

https://github.com/saddam213/OnnxStack/discussions/categories/ideas

[JohnClaw]

But to convert the whole project now to OpenVino for a few seconds seems overkill

I have only a laptop with a cheap CPU (no GPU is onboard), so for me OpenVino + LCM is much faster than ONNX + LCM. But i don't like Python so i prefer to use OnnxStack, though it slower due to lack of OpenVino. :(

[saddam213]

My entire project is build on Onnx so unfortunately that wont be an option here.

However FastSdcpu looks pretty awesome, so no need to replace it

[NeusZimmer]

Inpainting prototype is kinda working, small edits seem to work, but large edits seem to just grey out

Still has some work to go, but if you want to test it you will need to use the inpainting model https://huggingface.co/runwayml/stable-diffusion-inpainting (this wont work on txt2img or img2img)

If you know of some good repos that would be useful let me know, always good to have more insight

HI, you asked for good repos... not sure if good, but maybe you may look into my rep. Being working on another UI by myself, got main pipelines (lcm working on test env but no uploaded)....and a couple of my own ideas that might be useful (i.e. disk space and memory management for low mem cards...)