search

axodox / axodox-machinelearning

This repository contains a pure C++ ONNX implementation of multiple offline AI models, such as StableDiffusion (1.5 and XL), ControlNet, Midas, HED and OpenPose.
MIT License
605 stars 35 forks source link

Has iOS support and what model is supported? #22

Open paulocoutinhox opened 11 months ago

paulocoutinhox commented 11 months ago

Hi,

First, nice project.

  1. It has iOS/Android support?
  2. What i need to run in iOS?
  3. And what models are supported?
  4. Can i use safetensors or ckpt models?

Thanks.

axodox commented 11 months ago
  1. Not yet, admittedly speed and/or resolution would be limited on a mobile chip, or would take quite long to generate an image.
  2. Would require some work, but should not be very substantial as the base code is written in a platform agnostic way. You would need to configure the sessions to use an IOS compatible execution provider, and port a little code (e.g. mainly image loading, saving) to the OS.
  3. Currently SD 1.5 works with all basic features (txt2img, img2img, inpainting) and controlnet. SD XL support is in progress on the sdxl branch, basic image generation works, but quality is not perfect yet. I do not think you will run SDXL on a phone though, it is quite heavy for that.
  4. The lib uses ONNX models. However I wrote a guide for conversion: https://github.com/axodox/unpaint/wiki/Model-import. The upcoming version of the ONNX converter respects the weight tensor names, so with that it will be possible to load model weights from safetensors and other formats.