ipywidgets/gui from VSCode?

[ghostsquad]

I'm not that familiar with Jupyter Notebooks. Is it possible to get the images / simple UI you have in the README straight from VSCode? Do you have any other recommendations/reading around that?

[Chaoses-Ib]

Yes, the screenshots in README are all from VS Code Notebook. When you open examples/runtime.ipynb in VS Code, it will prompt you to install the Notebook extension if you haven't. After then it should be straightforward to use.

[ghostsquad]

Awesome! Thanks for the help. I feel like there should be some instructions for this in the readme or something like that. I'll try that this morning.

[ghostsquad]

I've spent way too long outside of Python land... and 0 experience with Jupyter. I have a couple of questions.

1. How do I get navigation to work (i.e. go to function) (notice the yellow squiggles)?
2. This question may be answered once I get an answer to my first question. Why doesn't this example workflow work? I don't see why CheckpointLoaderSimple would not have 3 values returned.

I wish more stuff was written in Golang or something like that. :(

[Chaoses-Ib]

For some reason, your src\comfy_script\runtime\nodes.pyi is not properly generated or loaded. Normally it should look like this:

You can check if nodes.pyi is there, and if it is, restarting VS Code may solve the problem.

CheckpointLoaderSimple should have 3 values returned. I'm not sure about your case. Maybe some special nodes hacked it and changed its return value. You can print(CheckpointLoaderSimple('...')) to see what are actually returned.

[Chaoses-Ib]

I wish more stuff was written in Golang or something like that. :(

If you only need to use the API (virtual mode), I remember there is a Go library similar to what ComfyScript does. Though I'm not sure if it supports arbitary custom nodes and is still maintained.

[ghostsquad]

Oh, that's right, I have a locally modified version of this to also return the checkpoint name as a string, so I can make decisions about that automatically.

[ghostsquad]

For some reason, your src\comfy_script\runtime\nodes.pyi is not properly generated or loaded.

That file does not exist. I have ComfyUI installed using the "portable" method, and when loading up examples/runtime.ipynb I pointed the python interpreter at "E:\ComfyUI_windows_portable\python_embeded\python.exe".

I'm probably missing something in regards to how ComfyScript is supposed to be used/installed. I feel like the README here glosses over many things and/or makes a lot of assumptions.

I appreciate your time in helping out. I'd be happy to contribute back to this as well, once I get this all working.

API (virtual mode)

Talking about just an automated way of generating a workflow and submitting it to the API?

[Chaoses-Ib]

That file does not exist.

That's strange. If you want, an ad-hoc way to fix it is to go to the source code of load() (pressing F12) and change Path(__file__).resolve().with_suffix('.pyi') to a fixed path, like r'E:\nodes.pyi', and then move it to the same directory of nodes.py.

I'm probably missing something in regards to how ComfyScript is supposed to be used/installed. I feel like the README here glosses over many things and/or makes a lot of assumptions.

I did assume the user to have some Python experience when writing the docs, mainly because using this would be harder than the web UI for people don't know Python well and thus doesn't make much sense. There are many problems reported in issues not mentioned in docs too. I have to save time to write the actual code. This is truly a bad thing for new users.

Talking about just an automated way of generating a workflow and submitting it to the API?

Yes. Otherwise one has to use Python to call ComfyUI.

[ghostsquad]

It's been many years since I was writing Python for my day job. I'm a software engineer, but mostly working in Go. So it's not like I'm completely lost. In fact, I came here because I'm frustrated with some of the tedium involved in using the Comfy UI. The Node spaghetti, the constant panning and zooming. The inability to quickly spot differences in various parameters. The difficulty in making runtime decisions, etc. I want to simplify my workflows, and I think that comes in the form of concise code.

[Chaoses-Ib]

That's exactly the reason I wrote this project. Many other users are using this for writing backends, but for me, It's always for writing workflows in code instead of connecting nodes in a graph GUI.

[ghostsquad]

oh...

comfy_script/runtime/nodes.py but no node.pyi here.

comfy_script/runtime/real/nodes.py as well as comfy_script/runtime/real/nodes.pyi exists.

[ghostsquad]

I went straight to running real instead of the ComfyScript Runtime at the top of runtime.ipynb. Now I have comfy_script/runtime/nodes.pyi

[ghostsquad]

I also just discovered this: https://stackoverflow.com/questions/53578365/does-vscode-support-python-pyi-files-for-intellisense - Gotta go change the python language server setting in VSCode from default to Pylance.

[Chaoses-Ib]

Oh, I did only test it with Pylance. I thought VS Code's Python extension only works with Pylance, but it turns out it also supports Jedi, although Pylance is installed by default. I will add a troubleshooting in README.

[Chaoses-Ib]

I've also added a .vscode/settings.json to specify the use of Pylance. Although it won't work outside of the repo, I hope it can make things clearer.

[ghostsquad]

I was able to get this working yesterday! 🎉 Now I gotta go through and start the process of grabbing the transpiled workflows and clean up the code.

I think the things I'm going to run into issues with quickly creating masks. I don't recall if you mentioned a solution for adhoc drawing of masks. I don't use Photoshop, instead I use Affinity Photo.

[Chaoses-Ib]

For Photoshop, there are two ways, one is to call its API to wait for changes and then retrieve the image, the other one is to watch for file saves and then convert the saved PSD to image. I don't know if any of them is applicable to Affinity Photo. A quick search shows me that .afphoto contains PNG copy in it: https://github.com/immich-app/immich/discussions/4022 , so the latter approach could work, but it may be unpleasant if saving files is slow in Affinity Photo.

Another solution is to draw the mask in Jupyter Notebook. It's possible in theory, but lacks good implementation in practice. The following is an example from ipycanvas. It works, but is laggy and lacks common features of an image editor.

There is also anywidget and ipyreact for packaging web components into Jupyter Notebook widgets. But this requires knowledge of the frontend and possibly costs a lot of time.

[ghostsquad]

I'd be fine with a file watcher. Even if Affinity takes 3 seconds to save. I'll look into this. Thanks for that link.

[ghostsquad]

It's funny, now that this is running, I'm starting to get a feeling that developing my own nodes that are composites of other nodes may work better. The workflow I have is insane, but it's mostly all logic, like if statements, and muting and such, which is a headache. It clutters the screen, and I really want to hide it all. I'm starting to see that is what a lot of people are doing, making their own nodes which combine a lot of runtime logic to reduce spaghetti. What's your take?

[Chaoses-Ib]

I generally avoid doing so unless the encapsulated function has a clear boundary and is flexible enough. Otherwise, it may need to change frequently and break existing code. That's only my option though.