How to remove prompt in promptless CN

[KhawlahB]

I am trying to do promptless training for ControlNet.

I am wondering what is the correct way to remove the prompt during model training on my dataset.

I need to train it for image2image task with conditional image

Should I pass empty string here?

or should i remove it completely from RETURN statement?

@williamyang1991 @lllyasviel @eltociear @camenduru

[geroldmeisinger]

Original ControlNet paper: "In the training process, we randomly replace 50% text prompts ct with empty strings. This approach increases ControlNet’s ability to directly recognize semantics in the input conditioning images (e.g., edges, poses, depth, etc.) as a replacement for the prompt."

[KhawlahB]

I have read this, but how to do it? should i provide a json file with 50% empty string?

Also, i want to experiment when all prompts are empty, do not know if that possible using the way that i have sent? @geroldmeisinger

[geroldmeisinger]

line 23: prompt = item['prompt'] if random() >= 0.5 else "" (updated: correct python)

[KhawlahB]

this line of code gonna replace 50% by random string? what is the purpose of using (random) number here? Can you pleas explain it in words, what the code you have written trying to do?

Also, if i am trying to remove the prompt completely-> replace 100% of prompts by empty string! can i do the same by changing the number from 0.5 to 1.0? i want to see the results if i remove the prompt...

@geroldmeisinger @williamyang1991 @scarbain @lllyasviel

[geroldmeisinger]

50:50 chance to use empty string

how long does it take for you to train a controlnet? couldn't you just try? did you already successfully train a controlnet without "extras"?

[KhawlahB]

• actually it takes long time each epoch takes about 6h, so i stopped it with epoch25. I do not know why it takes this a lot of time? despite that i am using GPU

• How many training epochs does Controlnet have (total number)?

• So i can understand that, if i replace 0.5 with 1.0 that means 100% chance of using empty string? please confirm for me if i understand you correctly.

@geroldmeisinger

[yanhn]

@KhawlahB Hi， Any conclusion of the question?

Also, if i am trying to remove the prompt completely-> replace 100% of prompts by empty string! can i do the same by changing the number from 0.5 to 1.0? i want to see the results if i remove the prompt...