search

nixified-ai / flake

A Nix flake for many AI projects
GNU Affero General Public License v3.0
656 stars 69 forks source link

Support for ComfyUI? #84

Open RyzeNGrind opened 7 months ago

RyzeNGrind commented 7 months ago

There is a docker image with InvokeAI, A1111, and ComfyUI. Is it possible to make this flake have support for running a nix native dockerImagebuild of the referenced URL so we can run any of the 3 via a flake? While maintaining NixOS guest machines, WSL machines, and hypervisor hosts?

max-privatevoid commented 7 months ago

Docker would be a pretty heavy dependency to introduce, and just fetching a Docker image from elsewhere would defeat the point of this project.

Packaging ComfyUI itself as per the title should be doable, although InvokeAI provides very similar node graph functionality.

RyzeNGrind commented 7 months ago

Docker would be a pretty heavy dependency to introduce, and just fetching a Docker image from elsewhere would defeat the point of this project.

Packaging ComfyUI itself as per the title should be doable, although InvokeAI provides very similar node graph functionality.

Sorry I should have been more specific in my issue above. Perhaps we could use something similar to dockerTools to bring reproducibility to the nixified-ai flake when working with other docker images?

Or like you mentioned instead of relying on docker, perhaps we could integrate comfyui, invoke is already here, maybe we can bring in A1111 as well?

I havent used InvokeAI yet myself, I jumped straight into ComfyUI as it seemed to be the most capable of producing the best outputs when combined with complex workflows.

max-privatevoid commented 7 months ago

We could use dockerTools to reproducibly create our own Docker images, but I don't see how it'd help with making other Docker images more reproducible. All we can do with them is fetch them.

I'm definitely not against adding ComfyUI. For A1111 I'm a bit more cautious as I remember it being pretty messy, especially with its plugin system. Of course I'm happy to be disproven.

Take a look at InvokeAI's node graph mode, it's basically a clone of ComfyUI and should feel very familiar to you. Maybe you'll like it.

RyzeNGrind commented 7 months ago

Hmm, it definitely does look promising to use the InvokeAI's node graph mode. Perhaps I will try installing via this flake and test it out? I have a Nvidia GPU running on my Windows 11 PC with WSL so will try to see if I can get any output there.

I checked out the community plugins for Invoke, and it seems there is way more community support for ComfyUI and their custom nodes. If there is a way for adding in ComfyUI in a relatively low effort frictionless manner, I think the community would benefit from bringing all those OSS ComfyUI CustomNodes over to a Nix compatible enviroment with ComfyUI running natively in NixOS.

BatteredBunny commented 6 months ago

Hi i have packaged comfyui in my own nix ai repo, feel free to take a look at it https://github.com/BatteredBunny/nix-ai-stuff