WASasquatch / was-node-suite-comfyui

An extensive node suite for ComfyUI with over 210 new nodes
MIT License
1.15k stars 170 forks source link

Mask batch #10

Closed AbyszOne closed 1 year ago

AbyszOne commented 1 year ago

Is there any way to use mask in the batch? Also, is image to latent mask the node to use masks or there is no SD mask node in this suite. Base mask node doesnt allow to input batch images.

WASasquatch commented 1 year ago

I don't think SD even allows batch init. I am not sure what you mean. And Image to Latent Mask allows you to inpaint yes. You have to use the mask latent noise node

AbyszOne commented 1 year ago

I was referring to using a batch of masks like the one in the SD gui. Regarding the batches, it is a very useful function but the current implementation has several conflicts. One of the most obvious is that it uses the root folder to host the txt, and that prevents you from using two processes with the same folder, since the batches overwrite each other and add up. It is also not clear how it counts and resets the batches. I have resigned myself to deleting the txt after each round, because it does not restart after finishing the batch. In particular, I am using a workflow that uses the same images in multiple ways, so with the current system I have to create copies and copies of each folder so that the batches do not conflict. Edit: First conflict may solve changing the txt name, but still i dont think a txt inside the process folder is a good idea. There are some task that could count that txt as a file or so, and this includes external workflows.

WASasquatch commented 1 year ago

You do realize it saved the counter in the image folder you target, and you can name the image counter anything you want like counter1.txt and counter2.txt and use same folder.

And no, no tasks open txt files but my text file loader. Storing the text files is required and the best method because I do thousands of images and need to continue those batches. You can't store counters in the script because they will be lost every run, and storing in settings for comfy get restored every time you open it, so wouldn't be able to continue batches

AbyszOne commented 1 year ago

Well, it makes sense if it's for personal use and you don't expect to be friendly to the general user. Something that shouldn't be troublesome to change is to create the destination folder if it doesn't exist. It currently reports that it does not exist and stops. It would be very useful when doing multiple tests.

WASasquatch commented 1 year ago

Why would it create a destination folder that doesn't exist when it's looking to load files? It will have to error regardless because it can't return anything.

AbyszOne commented 1 year ago

Why would it create a destination folder that doesn't exist when it's looking to load files? It will have to error regardless because it can't return anything.

¿? A destination folder is a destination folder, not an input folder. Why would be a problem creating it? The opposite is the problem, if any.

WASasquatch commented 1 year ago

Why would it create a destination folder that doesn't exist when it's looking to load files? It will have to error regardless because it can't return anything.

¿? A destination folder is a destination folder, not an input folder. Why would be a problem creating it? The opposite is the problem, if any.

I thought we were talking about the load batch images node still.

AbyszOne commented 1 year ago

I see. Anw, good suite, more useful than default for sure.

WASasquatch commented 1 year ago

I see. Anw, good suite, more useful than default for sure.

Thank you. I just pushed patch so you can save to destination that doesn't exist: https://github.com/WASasquatch/was-node-suite-comfyui/commit/f73e06061be5c059f2fa5d939f27f454306cc36b#diff-35652c7be27f406523ce77f22809c807782fa83236d30eed7d170b3859941ddbR2054

WASasquatch commented 1 year ago

I see. Anw, good suite, more useful than default for sure.

Load Image Batch is updated and changed a bit. It now stores it's settings internally, but it also allows you to use a pattern glob for the directory. So you could use **/* to search the directory and all subdirectories, or *.jpg to only select jpg images in the directory. Or even *_mask.png

AbyszOne commented 1 year ago

Great. Ill send feedback soon.

AbyszOne commented 1 year ago

Seem there is looking for a second comfy folder that doesnt exist:

"File F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 133, in _save
    with open(self.filepath, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'F:\\AIUI\\ComfyUI/ComfyUI/custom_nodes\\was_suite_settings.json'

Must be clarified that also im using a github clone folder, not the raw WAS script, wich is the more practical way to be updated. Now, a little thing and a kinda important one: 1 - If you delete prefix in image save it still uses _ as prefix. Thats a bit problematic in some contexts. 2 - I think in some workflows could be very useful a image save mode allowing to change between incremental and overwrite. Particularly for batch processing and multi test rounds. Having only incremental default mode force you to create many unnecessary folders or manually delete the files if all you want is repeat some tests.

WASasquatch commented 1 year ago

It's checking for the clone folder and file, was-node-suite-comfyui. did you rename the folder or main file?

AbyszOne commented 1 year ago

F:\AIUI\ComfyUI/ComfyUI/custom_nodes\was_suite_settings.jso

No, just normal clone into scripts folders. This is the part im talking about F:\\AIUI\\ComfyUI/ComfyUI/custom_nodes\\was_suite_settings.json. Thats not a correct path.

AbyszOne commented 1 year ago

Full error:

 File "F:\AIUI\ComfyUI\execution.py", line 177, in execute
    executed += recursive_execute(self.server, prompt, self.outputs, x, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  [Previous line repeated 3 more times]
  File "F:\AIUI\ComfyUI\execution.py", line 65, in recursive_execute
    outputs[unique_id] = getattr(obj, obj.FUNCTION)(**input_data_all)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 709, in load_batch_images
    fl = self.BatchImageLoader(path, label, pattern)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 728, in __init__
    self.WDB.insert('Batch Counters', label, 0)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 114, in insert
    self._save()
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 133, in _save
    with open(self.filepath, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'F:\\AIUI\\ComfyUI/ComfyUI/custom_nodes\\was_suite_settings.json
WASasquatch commented 1 year ago

Full error:

 File "F:\AIUI\ComfyUI\execution.py", line 177, in execute
    executed += recursive_execute(self.server, prompt, self.outputs, x, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 56, in recursive_execute
    executed += recursive_execute(server, prompt, outputs, input_unique_id, extra_data)
  [Previous line repeated 3 more times]
  File "F:\AIUI\ComfyUI\execution.py", line 65, in recursive_execute
    outputs[unique_id] = getattr(obj, obj.FUNCTION)(**input_data_all)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 709, in load_batch_images
    fl = self.BatchImageLoader(path, label, pattern)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 728, in __init__
    self.WDB.insert('Batch Counters', label, 0)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 114, in insert
    self._save()
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 133, in _save
    with open(self.filepath, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'F:\\AIUI\\ComfyUI/ComfyUI/custom_nodes\\was_suite_settings.json

Can you try the latest push? We're now using os.sep to try and keep better path separator structure. And now the database is reading and writing in + mode which should create the file if it doesn't exist. Apparently normal w and r creates files on some windows system, but in general, isn't supposed to.

There is one other bit of logic I am unsure if is needed (adding blank {} dict after creation of empty file. I don't think it's needed.

As far as saving files I largely left the naming logic of the original node alone, as it's very weirdly put together (feel like a name could be determined while setting a var with shorthand then a function plus logic) and intertwined throughout the whole function. I'll take a look at it again with your considerations.

AbyszOne commented 1 year ago

I'd like to understand enough of the code to help, but I can barely get my script to work. What I can tell is that such a was_suite_settings.json does not exist in any folder, and the problem persists with the latest pull. I am not able to render anything.

WASasquatch commented 1 year ago

I'd like to understand enough of the code to help, but I can barely get my script to work. What I can tell is that such a was_suite_settings.json does not exist in any folder, and the problem persists with the latest pull. I am not able to render anything.

What OS and venv environment are you running? It creates the file on its own, and does in all my tests on windows 11 and Ubuntu. It's now in w+ and r+ mode which create the file if it doesn't exist, so not much I can do beyond that. You can create the file yourself and add {} to it.

I just tried removing the try/catch statement. But according to your error it occurred during writing.

Another strange thing is you say you're cloning the repo, but it says you aren't using the cloned repo because this line

WAS_SUITE_ROOT = ( CUSTOM_NODES_DIR if not os.path.exists(os.path.join(CUSTOM_NODES_DIR, 'was-node-suite-comfyui')) else os.path.join(CUSTOM_NODES_DIR, 'was-node-suite-comfyui') ) is resolving to custom nodes dir, not was-node-suite-comfyui because it says it can't be found. And if it exists, then there is something serious wrong with your venv or systems IO abilities.

The CUSTOM_NODES_DIR is resolved from CUSTOM_NODES_DIR = os.getcwd()+os.sep+'ComfyUI'+os.sep+'custom_nodes' which gets the working directory of comfyui

image

So maybe it's because you are not launching from this folder structure?

WASasquatch commented 1 year ago

Alright, I added some better file logic. It'll get the file paths based on the executing node file: https://github.com/WASasquatch/was-node-suite-comfyui/blob/main/WAS_Node_Suite.py#L44

AbyszOne commented 1 year ago

That solved the wrong path, not the .json issue. I create it and seems to be working. Ill check later.

AbyszOne commented 1 year ago

Talked too soon. It creates the output folder, wich is nice, and makes the picture, but didn`t saved the picture on:

st! Creating directory.
Traceback (most recent call last):
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 2071, in save_images
    [-1] == "_", map(map_filename, os.listdir(self.output_dir))))[0] + 1
FileNotFoundError: [WinError 3] El sistema no puede encontrar la ruta especificada: 'F:\\AIUI\\ComfyUI\\ComfyUI\\output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "F:\AIUI\ComfyUI\execution.py", line 177, in execute
    executed += recursive_execute(self.server, prompt, self.outputs, x, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 65, in recursive_execute
    outputs[unique_id] = getattr(obj, obj.FUNCTION)(**input_data_all)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 2075, in save_images
    os.mkdir(self.output_dir)
FileNotFoundError: [WinError 3] El sistema no puede encontrar la ruta especificada: 'F:\\AIUI\\ComfyUI\\ComfyUI\\output'

For instance, my output was nothing remotely close to that path.

WASasquatch commented 1 year ago

Your system is doing weird stuff to paths lol

Talked too soon. It creates the output folder, wich is nice, and makes the picture, but didn`t saved the picture on:

st! Creating directory.
Traceback (most recent call last):
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 2071, in save_images
    [-1] == "_", map(map_filename, os.listdir(self.output_dir))))[0] + 1
FileNotFoundError: [WinError 3] El sistema no puede encontrar la ruta especificada: 'F:\\AIUI\\ComfyUI\\ComfyUI\\output'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "F:\AIUI\ComfyUI\execution.py", line 177, in execute
    executed += recursive_execute(self.server, prompt, self.outputs, x, extra_data)
  File "F:\AIUI\ComfyUI\execution.py", line 65, in recursive_execute
    outputs[unique_id] = getattr(obj, obj.FUNCTION)(**input_data_all)
  File "F:\AIUI\ComfyUI\custom_nodes\was-node-suite-comfyui\WAS_Node_Suite.py", line 2075, in save_images
    os.mkdir(self.output_dir)
FileNotFoundError: [WinError 3] El sistema no puede encontrar la ruta especificada: 'F:\\AIUI\\ComfyUI\\ComfyUI\\output'

For instance, my output was nothing remotely close to that path.

That was a little more straight forward of an issue haha. That should be fixed in latest push. Additionally, you an use the prefix as a filename if overwrite is true: https://github.com/WASasquatch/was-node-suite-comfyui/blob/main/WAS_Node_Suite.py#L2037

AbyszOne commented 1 year ago

Fingers cross, my system is working on many repos without such behavior yet. Ill test updates asap.

WASasquatch commented 1 year ago

Fingers cross, my system is working on many repos without such behavior yet. Ill test updates asap.

Yeah, this is why people probably use pathlib so they don't have to worry about separators and all that stuff, so it just works on any systems file structure.

WASasquatch commented 1 year ago

I also went through and made the auto install process be a little smarter about where it's looking and what it's doing. File permissions though is a user-thing. People will have to make sure their ComfyUI custom-nodes dir and files are writable.