Can other model be used in the android apk?

[Fcucgvhhhvjv]

Hi developer thanks for making this awesome app, i recently got to know about stable diffusion so I was curious to run it locally, I've found that it can work on pc, mac, ios but not android. I finally found it after all.

Can we use other models if so how, for example I want to use cyber realistic model.

[ZTMIDGO]

Yes, you can follow https://github.com/ZTMIDGO/Android-Stable-diffusion-ONNX/issues/6 to convert SD models to ONNX

[Fcucgvhhhvjv]

Thanks for responding, ill try to convert cyber realistic and update it here. So far the app is working better than expected on my Poco x3 pro 6GB, model usages is around 2-2.5G. But it crashes when i try to generate image more than 384 x 384.

[Fcucgvhhhvjv]

Hi, I tried to convert a model epic_realism to do so I used sdtoonnxf16.py but that conversion stopped after giving the unet file it was model.onnx and weight.pb, I tried to do this with 3 models but all of it stopped after giving unet. I think all other files in the model directory is same for all the model just the model.ort is different. So I quantized the onnx.model and i got a onnx.model.data I renamed it to model.ort. Since I don't have a pc and i cant build the app I tried to unzip the apk and replace the model.ort with the one I got the I resigned the app and tried to install it but I got an error, it says app is invalid. I can't test the model myself, can you check if its correct. Let me know I'll update with a link to download it.

[Fcucgvhhhvjv]

Hi, I tried to convert a model epic_realism to do so I used sdtoonnxf16.py but that conversion stopped after giving the unet file it was model.onnx and weight.pb, I tried to do this with 3 models but all of it stopped after giving unet. I think all other files in the model directory is same for all the model just the model.ort is different. So I quantized the onnx.model and i got a onnx.model.data I renamed it to model.ort. Since I don't have a pc and i cant build the app I tried to unzip the apk and replace the model.ort with the one I got the I resigned the app and tried to install it but I got an error, it says app is invalid. I can't test the model myself, can you check if its correct. Let me know I'll update with a link to download it.

[ZTMIDGO]

Of course, can you show me the converted code or project?

[Fcucgvhhhvjv]

This is roughly the same colab I used to convert https://github.com/Fcucgvhhhvjv/Android-Stable-diffusion-ONNX/blob/master/Sdtoonnx.ipynb .

After the unet model gets converted the script gives some error like this _model_to_graph graph = _optimize_graph( File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 665, in _optimize_graph graph = _C._jit_pass_onnx(graph, operator_export_type) File "/usr/local/lib/python3.10/dist-packages/torch/onnx/utils.py", line 1901, in _run_symbolic_function raise errors.UnsupportedOperatorError( torch.onnx.errors.UnsupportedOperatorError: Exporting the operator 'aten::scaled_dot_product_attention' to ONNX opset version 17 is not supported. Please feel free to request support or submit a pull request on PyTorch GitHub

https://drive.google.com/drive/folders/1mk7C0Qcqno0424zT0pUamxY7mJDhMKMP

This is what I got. I'll try to compile the app in termux but i dont think it will work.

[ZTMIDGO]

I can't use netron to see the input shape of the model https://drive.google.com/drive/folders/1mk7C0Qcqno0424zT0pUamxY7mJDhMKMP , if your model is consistent with the input shape of the model I use, I think the problem should be solved

[Fcucgvhhhvjv]

I have model.onnx.data and model.onnx so first I have to convert it to model.ort then change the model with the one I got? But i cant build the app as I don't have a pc , I tried to do it with termux gradle but it crashes halfway due to ram I think, can u build the app with this model if i get the model.ort . Im not sure if the model conversion will work as it failed halfway but unet conversion was successful.

[ZTMIDGO]

I haven't used termux gradle, and I don't know much about this tool. If you need to build an App, I suggest using Android studio

[Fcucgvhhhvjv]

I can't use android studio because I don't have a computer, I tried to repack the app and replace my model and resign it but It said invalid app, i did the same for some other app and it worked why is it not working with this? Also can I load this binary in a linux arm 64 environment like txt2img.py so I can test models easily?

[Fcucgvhhhvjv]

HI, I was able to build everything and got the app working too, I used github ci android BUILDER. can u make this model and apk available so others can use it? I used the model epicrealism. Here is the result https://huggingface.co/spaces/Duswishdb/Xc/resolve/main/Screenshot_2023-07-22-21-14-08-086_com.example.open.diffusion.jpg

[ZTMIDGO]

Looks good, although I don't quite understand how you compiled the code using github ci android BUILDER

[Fcucgvhhhvjv]

Ohh u can check android.yaml in my repo which I forked from yours. Should i provide you with the link to apk file and the model.ort Also can u implement save image, sampler and use model without having to rebuild the app in the next update . Binary to run it from linux environment and

[Fcucgvhhhvjv]

https://github.com/Fcucgvhhhvjv/Android-Stable-diffusion-ONNX/blob/master/.github/workflows/android.yml

[ZTMIDGO]

Sure， I will update the README.md

[Fcucgvhhhvjv]

Here is the app that use epicrealism from civit ai https://huggingface.co/spaces/Duswishdb/Xc/resolve/main/app-debug.signed.apk And the result https://huggingface.co/spaces/Duswishdb/Xc/resolve/main/Screenshot_2023-07-22-21-52-06-093_com.example.open.diffusion.jpg

And the model used https://drive.google.com/drive/folders/1RgfeXX6hcew3ynJQWMq4cKDPUAtgYuai

[ZTMIDGO]

ok

[Fcucgvhhhvjv]

Can u add sampler save the image and load models from storage instead of packing in the app, thank you for your hard work, I'll keep updating if i convert another model

[ZTMIDGO]

Of course, I will modify the code in the next few weeks, currently I am focusing on the porting of rwkv midi, and soon can generate midi music through Android

[Fcucgvhhhvjv]

Ohh I'm looking forward to it, I'm new to coding and all, but I'll like to contribute In some way.

[Fcucgvhhhvjv]

I have converted 2 more models and built the apk but output are not always good , what is the default scheduler it's running is there a way to change it to DPM cuz there is a file name core having scheduler so I'm wondering if it's possible to use it . could the failed conversion of model folders like vae tokenizer and others except unet result the bad image sometimes ?