CoreML Diffusers?

[honzajavorek]

I've noticed that there's now a new way how to speed up image generation on macOS. Is this something DB already does, or is it something that could be incorporated?

• repo
• tweet

[gingerbeardman]

Requires macOS 13 (Ventura or later)

[honzajavorek]

I got it working, but in the end, I wasn't able to get anything interesting out of it. Felt like 80 % hallucinating, also no custom models.

[LeaveNhA]

Is this because you are not familiar with the methods and tools or we can't use custom models or do any customisations?

[honzajavorek]

So I explore the possibilities and my I-dont-know-what-I'm-doing take is:

• The tweet links to a particular app, but the technology is app-independent.
• The technology is called CoreML
• The particular app doesnt allow for custom models, but in general, you can use custom models with this technology, you just need to convert them to a particular format
• There are scripts out in the wild to convert models to diffusers and then to this particular format
• There are other apps similar to DB which build on top of this technology, such as Mochi Diffusion
• The technology limits the prompt to 75 tokens, doesn't allow to choose the sampler or the resolution
• The thing produces very bad results compared to what one can get out of automatic webui - basically, when I tried the same model, the same number of steps, a similar prompt, I've been getting hilariously funny results from CoreML while "the Python way" has provided me with serious results really sticking to the prompt (e.g. prompting for a mermaid resulted in getting a bearded guy in CoreML, while webui provided what I'd expect)
• It's very fast, yes! But what it's good for, if the results are laughable?