Nourepide / ComfyUI-Allor

ComfyUI plugin for image processing and work with alpha chanel.
MIT License
211 stars 26 forks source link

ImageTextMultilineOutlined + Text Load Line From File #16

Closed jgal14 closed 10 months ago

jgal14 commented 10 months ago

Hi there,

At the moment there doesn't appear to be a way to create multi-line text from a file (perhaps I'm missing an obvious solution).

Is it possible to add the ability to read \
in text, or to set an automatic line break feature at a set character / word / container width?

Thanks!

Nourepide commented 10 months ago

So you need a node that allows you to read text from a file?

And you also need a node that allows you to set a fixed width?

I apologize if I'm misunderstanding, I've been writing documentation all night and now it feels like my brain is literally drying up.

Nourepide commented 10 months ago

On a separate note, one piece of feedback (or potentially user error) is that a useful feature on the image resize nodes would be a 'max height' or 'max width' - I.E resizing to a max height or width dimension whilst maintaining existing aspect ratio.

If it's not too much trouble here too it could use more details for my tired brain. 😅

I suppose it will take me not so much time to realize something with image resizing.

jgal14 commented 10 months ago

Hey there,

Apologies I should have been clearer.

I have a node to load text from a file, however it loads a single line at a time from a txt file - Which means the multi-line text node only generates text in a single line (without any line breaks).

The goal is to be able to load a single line of text, and somehow set the line-break point.

I'm sure there are multiple ways to address this functionality.

My thoughts were;

Let me know if you would like further clarification.

Thanks

Nourepide commented 10 months ago

Okay, I think I'm starting to get it.

However, I do have a few questions:

Your node only reads one line from the file, does this mean that after concatenating the lines, the line break information is erased? Or is it iterated over the lines each queue?

jgal14 commented 10 months ago

On a separate note, one piece of feedback (or potentially user error) is that a useful feature on the image resize nodes would be a 'max height' or 'max width' - I.E resizing to a max height or width dimension whilst maintaining existing aspect ratio.

If it's not too much trouble here too it could use more details for my tired brain. 😅

I suppose it will take me not so much time to realize something with image resizing.

Sure thing.

The 'constrain image' node from here contains the functionality I'm describing.

In the below example, I have set the max width to an arbirary ceiling (2000) and am controlling the output width of the image with the height (758). The output image maintains the aspect ratio of the original.

The use case here is to overlay the resized image onto another image (of which I know the dimensions), so having a max width / height limited allows for scaling the image to fit more easily.

image

I hope that description helps.

jgal14 commented 10 months ago

Okay, I think I'm starting to get it.

However, I do have a few questions:

Your node only reads one line from the file, does this mean that after concatenating the lines, the line break information is erased? Or is it iterated over the lines each queue?

The node only reads a single line, it doesn't offer a concatenate functionality.

I suppose using a concatenate node to combine lines before passing to the multi-line tool would be another way to solve this challenge!

Here is the node

image

I will test concatenating the text before feeding it into the multi-line node.

Thanks!

Nourepide commented 10 months ago

The 'constrain image' node from here contains the functionality I'm describing...

So if I'm reading this right:

If I understand correctly, tomorrow I will make a node called ImageTransformResizeClip with this functionality.

Nourepide commented 10 months ago

The node only reads a single line, it doesn't offer a concatenate functionality.

Is this your personal node that you made yourself?

If so, wouldn't it be easier to make a separate node that just loads the text from the file completely and puts newline wildcards in it?

I can help you with that.

jgal14 commented 10 months ago

The 'constrain image' node from here contains the functionality I'm describing...

So if I'm reading this right:

  • You get images of unknown size as input.
  • You expect the output to be images with height no greater than MAX_HEIGHT and width no greater than MAX_WIDTH.
  • In case the image exceeds one of these values it will be scaled to the minimum size of these values

If I understand correctly, tomorrow I will make a node called ImageTransformResizeClip with this functionality.

Yes, that sounds correct!

(just a caveat on the 3rd point, the output image must maintain it's original aspect ratio)

Excellent, thank you very much for your support.

Nourepide commented 10 months ago

(just a caveat on the 3rd point, the output image must maintain it's original aspect ratio)

Of course, that's what I thought right away.

It's just that I've been working with arrays for a long time and I'm used to calling size the union of width and height.

jgal14 commented 10 months ago

The node only reads a single line, it doesn't offer a concatenate functionality.

Is this your personal node that you made yourself?

If so, wouldn't it be easier to make a separate node that just loads the text from the file completely and puts newline wildcards in it?

I can help you with that.

It isn't my own node, it's from the WAS node suite.

I'm just testing the concatenate solution, and it appears to be working - thank you for the offer of support.

I do think a more advanced 'line-break' / Text wrapping functionality would be useful for formatting text - For example if you have a focus image on a page which you want text to be wrapped around. I suspect that's a functionality of a completely different scope though!

Nourepide commented 10 months ago

What scope of functionality is not that important. If you need a node to help you with something, I will always find time to make it for you.

You just need to describe very precisely what you want and it will definitely be realized by me.

jgal14 commented 10 months ago

What scope of functionality is not that important. If you need a node to help you with something, I will always find time to make it for you.

You just need to describe very precisely what you want and it will definitely be realized by me.

Thank you for your support.

I don't have a fully formed idea of what the 'perfect' text formatting functionality might be yet - I will think about it.

One very useful functionality would be the ability to have static text with dynamic words that can be imported from a txt file.

For example;

The man's name was [index 1], he lived at [index 2]. His favourite sport was [index 3] and his favourite team was [index 4].

Is that sort of thing even possible?

Nourepide commented 10 months ago

Is that sort of thing even possible?

Yes, of course. Especially after nodes in the Lens submodule I don't think it will be difficult like this.

But node can really be as complex in design as it is simple. I propose to do a brainstorming, which usually helps me to design complex functionality.

  1. What problem does it solve? - A brief description of what this node is for.
  2. What should it look like? - Approximate input and output parameters.
  3. How should it work? - Your idea of how the node should work.
  4. How should it be implemented? - There is no point in answering this part, it's completely my task, I just wrote it to tell you about my approach.

I suggest to start with the first question and the general picture will start to emerge.

Nourepide commented 10 months ago

Node ImageTransformResizeClip was implemented in 4c6099c.

Documentation - Resize Clip

Example of work:

image

jgal14 commented 10 months ago

Node ImageTransformResizeClip was implemented in 4c6099c.

Documentation - Resize Clip

Example of work:

image

Fantastic, I will test this later in my workflow!

jgal14 commented 10 months ago

Is that sort of thing even possible?

Yes, of course. Especially after nodes in the Lens submodule I don't think it will be difficult like this.

But node can really be as complex in design as it is simple. I propose to do a brainstorming, which usually helps me to design complex functionality.

  1. What problem does it solve? - A brief description of what this node is for.
  2. What should it look like? - Approximate input and output parameters.
  3. How should it work? - Your idea of how the node should work.
  4. How should it be implemented? - There is no point in answering this part, it's completely my task, I just wrote it to tell you about my approach.

I suggest to start with the first question and the general picture will start to emerge.

  1. What problem does it solve?

The functionality will allow users to create fixed text strings, with placeholder content to allow for personalisation to be callled from a file.

Something similar to the functionality shown here

I would suggest for flexibility the tokens are simply named 'index 1', 'index 2' etc and call the assiciated line on a text file.

I hope this helps to describe the functionality better?

Nourepide commented 10 months ago

As a result, the node will take two text values.

In general, the work of this node will be similar to printf from the C language.

This node will be part of the new Text module in Allor 2 (which will include data proccesing), so we'll have to wait.

jgal14 commented 10 months ago

As a result, the node will take two text values.

  • The first text value will contain parameters to populate the second text.
  • The second text value will contain text with places that can be replaced.

In general, the work of this node will be similar to printf from the C language.

This node will be part of the new Text module in Allor 2 (which will include data proccesing), so we'll have to wait.

Understood, thanks for the update!

On another note - Do you know of any models for the ImageSegmentation node that can segment just the head of a subject (without neck / upper body clothing)?

Thanks!

Nourepide commented 10 months ago

Hmm, I don't recall there being anything similar among the ONNX models.

However, there are "true" Image Segmentation models on Hugging Face that split images into multiple parts with a description of each part.

Among them there is actually one model that splits the head into several parts, including - face, eyes, ears, etc.

There are also models that find an object by its description.

None of this is supported yet, because ImageSegmentation node should actually be called ImageSegmentationRembg or ImageSegmentationOnnx.

I think it would be good to completely redesign the ImageSegmentation module in Allor 2.

jgal14 commented 10 months ago

Hmm, I don't recall there being anything similar among the ONNX models.

However, there are "true" Image Segmentation models on Hugging Face that split images into multiple parts with a description of each part.

Among them there is actually one model that splits the head into several parts, including - face, eyes, ears, etc.

There are also models that find an object by its description.

None of this is supported yet, because ImageSegmentation node should actually be called ImageSegmentationRembg or ImageSegmentationOnnx.

I think it would be good to completely redesign the ImageSegmentation module in Allor 2.

Thanks for the ideas!

I tracked down a some SAM image segmentation solutions and implemented them into my workflow, they seem to work well.

I had some further thoughts on text formatting functionality - Would you like me to open a new thread and note my ideas there?

Nourepide commented 10 months ago

Would you like me to open a new thread and note my ideas there?

Uh, sure. Feel free to suggest any ideas.

Nourepide commented 9 months ago

Is that sort of thing even possible?

Literally just finished finalising a similar function. It takes the text, finds the necessary key characters in it and modifies the text following these rules.

frb - Foreground Red Bright bgn - Background Green Normal au - Attribute Bold au - Attribute Underline

This function will be used for the new logging system in Allor v.2. image