When I click the Assets button it flashes a loading icon for a moment then nothing happens, I can't click Generate. I already put the model in assets folder, what am I missing?

[ReEnMikki]

The Setup and Generate buttons are greyed out, and the Assets button doesn't work

[ZTMIDGO]

You need to click here

[ReEnMikki]

You need to click here

When I click on that button it quickly / briefly displays a loading circle very quickly (for like 1/4 of a second) then it disappears, and nothing happens. I clicked like 20 times, nothing happens. The other buttons are still grey and cannot be clicked. I already created assets folder in the main folder, and inside it, I put my quantized onnx model (fp16), with all the folder names and file names inside similar to your Chillout mix model on Huggingface. Please help!

[ZTMIDGO]

Do you see abnormal printing?

[ZTMIDGO]

You need to use android studio to build the project

[ReEnMikki]

You need to use android studio to build the project

Yes Im open in Android Studio and I built project and click Run to Run with an emulated Android device, I don't see abnormality, please can you help me by recording a short video showcasing how to set this up properly ? I am very interested in this and I really want to get it working please

[ZTMIDGO]

https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/blob/master/2.jpg

Does your assets look like this?

[ZTMIDGO]

https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/blob/master/2.jpg

Does your assets look like this?

You need to make sure that the model directory where assets are placed is the same as mine.

[ReEnMikki]

https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/blob/master/2.jpg Does your assets look like this?

You need to make sure that the model directory where assets are placed is the same as mine.

Bruv thank you... I see why it didn't work now... inside assets folder I put directly the folders like text encoder tokenizer unet vae encoder in there... When I should've put a "model" folder inside assets, then inside that model folder, put the 4 subfolders above... So the button is clickable now but when I try to generate it crashes after 10 seconds. I thought fp16 model is supposed to be lighter than fp32 ? Can you please enlighten me ? Maybe in the mean time I will try your model and report back whether it works

[ZTMIDGO]

The fp16 model will cause memory overflow and crash the app

[ZTMIDGO]

Model that requires uint8

[ReEnMikki]

The fp16 model will cause memory overflow and crash the app

ok thank you I will try to quantize the onnx fp32 version of the model again and report if there is any error.. Thank you so much for the help I appreciate it 🙏

[ReEnMikki]

The fp16 model will cause memory overflow and crash the app

Hello my original ckpt model is fp16 but when I convert it to diffusers I converted it to fp32 (it's around 4GB instead of 2GB). And then to onnx and quantized. But now it still crashes when I try to generate 🥲🥲🥲 Please help.

[ReEnMikki]

@ZTMIDGO I don't know why, I tried your ChilloutMix model, even it crashes app 🥲

[ZTMIDGO]

see https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/issues/6

[ReEnMikki]

see #6

Do I must convert to onnx and quantize model exactly using those files/ methods, and other methods will not work despite also getting the result of a quantized onnx model ?

[ZTMIDGO]

Yes, you need to make sure that the input and output types of the model are consistent with the model I used. You can use netron to view the model

[ZTMIDGO]

https ://onnxruntime.ai/docs/performance/model-optimizations/ort-format-models.html

[ReEnMikki]

@ZTMIDGO

Sorry, but I still don't understand why your model still crash. After several crashes of my models, I tried to see whether your Chillout model work or not. And it still crash. Please help. This is not problem of converting to onnx or quantizing model because I am only testing your model

[ZTMIDGO]

How much memory does your device have? You can try lowering the generated resolution

[ReEnMikki]

How much memory does your device have? You can try lowering the generated resolution

I tried generate lowest resolution 192x192 it still crashes with your model. My virtual device Pixel 3a API 34 settings are default, RAM: 1536MB, VM heap: 256MB, Internal Storage: 6144MB, Mutual core CPU 12

[ZTMIDGO]

： >=8GB memory

[ReEnMikki]

： >=8GB memory

Sorry, do you mean RAM or Storage ?

[ZTMIDGO]

Requires RAM >= 8gb

[ReEnMikki]

Requires RAM >= 8gb

Ok, the images now get generated IF I use your model, but I followed your tutorial to convert model to onnx then quantize, then make sure put the exact same folders and files into the assets like your model. BUT it still says Error. My model is originally fp16 ckpt, then i converted to diffusers 4GB (hopefully fp32? because afaik fp32 is 4GB and fp16 is 2GB). Then use your provided py script to replace in the cloned github repo to convert model to onnx, then quantized, then converted onnx to ort. In the output I get lots more models but I only picked the same folders and files similar. What went wrong ? Could it be that since the model was originally fp16 ckpt, converting to fp32 diffusers is still meaningless ?

[ZTMIDGO]

Use Netron to see the input and output of the model, and then take screenshots for me

[ReEnMikki]

Use Netron to see the input and output of the model, and then take screenshots for me

So we have 4 ort models, each in text encoder, unet, vae decoder and vae encoder. I opened each of them and exported the graphs as PNG, but when I open them to zoom in they look so blurry that is impossible to be comprehended. Can you please elaborate more on how do I look for the input and output of the model to take screenshot for you ? And also, which model are you referring to out of those I listed above ?

[ReEnMikki]

Use Netron to see the input and output of the model, and then take screenshots for me

Nvm, I see there is an option to export as SVG, and it looks very clear when zoomed in. Here I shared the SVG files to my Drive. Can you check SVG files ? https://drive.google.com/drive/folders/1SXYLEtwkKMj0DVBPZornimobXqvQ5tRr?usp=sharing

[ZTMIDGO]

Your unt model is something that looks like this

[ReEnMikki]

Your unt model is something that looks like this

Well, I'm practically blind when it comes to these technical layers of the model structure. Can you please elaborate how do I get the model working like yours ?

[ReEnMikki]

Your unt model is something that looks like this

Left is my model and Right is your model. I see the difference is that in timestep tensor, mine is float32 while yours is int32. I'm not tech savvy at this and I don't understand why it happened. Please guide me how can I fix this

[ZTMIDGO]

You need to modify this

[ZTMIDGO]

You can use https://github.com/Amblyopius/Stable-Diffusion-ONNX-FP16 to convert to onnx, and then download https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/files/11745190/conv_sd_to_onnx.txt Replace conv_sd_to_onnx.py in the project

[ReEnMikki]

You need to modify this

How should I modify that line to ? (What will the modified result look like ?) Sorry for so many questions from me, but I'm noob at this technical stuff

[ReEnMikki]

You can use https://github.com/Amblyopius/Stable-Diffusion-ONNX-FP16 to convert to onnx, and then download https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/files/11745190/conv_sd_to_onnx.txt Replace conv_sd_to_onnx.py in the project

I followed all the steps in issue #6 you linked, the only problem I have is that colab says it cannot find the onnxruntime version in the requirements.txt, so I had to download another version, other than that I followed every step carefully. I'm not sure whether that onnxruntime version affects this or not.

[ZTMIDGO]

I will upload a python about conversion to you tomorrow

[ReEnMikki]

I will upload a python about conversion to you tomorrow

Thank you! I'm looking forward to it 🫡

[ReEnMikki]

I will upload a python about conversion to you tomorrow

Hello, it's been 2 days. I'm wondering whether there has been any unexpected problem that occured ?

[ZTMIDGO]

Due to China banning access to huggingface, I am temporarily unable to download the model for testing the conversion, it may take some time

[ReEnMikki]

Due to China banning access to huggingface, I am temporarily unable to download the model for testing the conversion, it may take some time

Can you access Google Drive ? I can send you the model you need via my Drive. If not Drive, then maybe Mediafire ?

[ReEnMikki]

Due to China banning access to huggingface, I am temporarily unable to download the model for testing the conversion, it may take some time

Hey, I uploaded your model to gofile.io, which I think is not blocked in China: https://gofile.io/d/gGRuh1 Can you try downloading it to test ? And let me know whether the site works or it is still blocked. If blocked, I will find another site to host the model for you so you can download.

[ZTMIDGO]

Very good, I can download the model now

[ReEnMikki]

Very good, I can download the model now

Hello, how have you tested the model conversion ?

[ReEnMikki]

Very good, I can download the model now

I hope you will at least reply to me whether some unexpected errors or problems happened again, I will try to help; sorry for being impatient

[ZTMIDGO]

https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/files/13536842/diffusion-convert-FP.zip

Here's my conversion code, you can try using conv_sd_to_onnx.py or conv_sd_to_onnx_2.py and modify the --model_path in the code

[ReEnMikki]

https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/files/13536842/diffusion-convert-FP.zip

Here's my conversion code, you can try using conv_sd_to_onnx.py or conv_sd_to_onnx_2.py and modify the --model_path in the code

Thanks for the code After quantizing do I have to keep the model.onnx or model-quantize.onnx ? Also the quantization still leave weights.pb in unet folder. Why is this quantization code different from your recommendation in #6 ? And I don't get the ort model, instead after quantizing just have onnx. How do I get ort ? In the past I ran !python -m onnxruntime.tools.convert_onnx_models_to_ort {model_dir} Now this doesn't work. The conversion to ort has error

Edit: I deleted the model.onnx and renamed model-quantize.onnx to model.onnx, then the convert to ort command I mentioned worked. I also deleted the weights.pb, please let me know if I did any step wrong

[ReEnMikki]

Now I downloaded it and opened with Netron. Unfortunately the timestep tensor is still float32 instead of being int32 like your model. I don't know where I did wrong, please help. You have 2 versions of convert-to-ONNX code and quantization code, I used the first version of both of them. I installed onnx version 1.13.1 and onnxruntime version 1.15.1, I'm not sure if that affects the result or not.