[Bug]: Generation output is messy when using a prompt with lots of "weight" or keywords

Catmechanic457 commented 2 months ago

Checklist

[X] The issue has not been resolved by following the troubleshooting guide
[x] The issue exists on a clean installation of Fooocus
[X] The issue exists in the current version of Fooocus
[X] The issue has not been reported before recently
[ ] The issue has been reported before but has not been fixed yet

What happened?

When generating images with prompts that use a lot of keywords, weights i.e (raining:1.0) or many styles, the output becomes noisy/messy.

This can happen very suddenly with little change to the prompt.

Steps to reproduce the problem

generate the image using the json provided in Console Logs
change (trees:2.5) to (trees:1.5) and use seed 244799644407406348 to generate the control image

My outputs described below:

2024-06-12_14-13-12_3382.png (control image) with prompt landscape, [snow], (trees:1.5), cherry blossom, (mountains), (flowers:1.5)

generates just fine.

2024-06-12_14-14-05_5741.png (issue image) with prompt landscape, [snow], (trees:2.5), cherry blossom, (mountains), (flowers:1.5) - (note trees:1.5 becomes trees:2.5)

does not generate correctly.

To my knowledge, the image is not corrupted in any way, it's an issue with the generation process.

Control image: 2024-06-12_14-13-12_3382

Issue image: 2024-06-12_14-14-05_5741

What should have happened?

The issue image should be similar to the control image only with more trees.

What browsers do you use to access Fooocus?

Google Chrome

Where are you running Fooocus?

Locally

What operating system are you using?

Windows 10

Console logs

{
"prompt": "landscape, [snow], (trees:2.5), cherry blossom, (mountains), (flowers:1.5)",
"negative_prompt": "",
"prompt_expansion": "landscape, [snow], (trees:2.5), cherry blossom, (mountains), (flowers:1.5), surreal, highly detailed, sharp focus, cinematic light, beautiful, elegant, intricate, innocent, amazing composition, ambient, shiny, epic, great color, perfect complex stunning, inspired, vibrant colors, creative, winning",
"styles": "['Fooocus V2', 'Fooocus Semi Realistic', 'Fooocus Masterpiece', 'SAI Anime', 'Misc Kawaii', 'Light Cheery Atmosphere', 'SAI Digital Art', 'MRE Anime']",
"performance": "Speed",
"resolution": "(1152, 832)",
"guidance_scale": 6,
"sharpness": 2,
"adm_guidance": "(1.5, 0.8, 0.3)",
"base_model": "juggernautXL_v8Rundiffusion.safetensors",
"refiner_model": "None",
"refiner_switch": 0.5,
"clip_skip": 2,
"sampler": "dpmpp_2m_sde_gpu",
"scheduler": "karras",
"vae": "Default (model)",
"seed": "5322216529148521707",
"metadata_scheme": false,
"version": "Fooocus v2.4.3"
}

Additional information

The example is my attempt at a "minimal reproducible example" however I've encountered this issue many other times without using high weights

JulienZLQ commented 2 months ago

"styles": "['Fooocus V2', 'Fooocus Semi Realistic', 'Fooocus Masterpiece', 'SAI Anime', 'Misc Kawaii', 'Light Cheery Atmosphere', 'SAI Digital Art', 'MRE Anime']",

I think it's too many styles

Catmechanic457 commented 2 months ago

@JulienZLQ I understand that, but the styles only add to the prompt. If I remember correctly, lllyasviel said somewhere that Fooocus should be able to handle long prompts. I could be wrong but I still believe it's an "issue" rather than user error.

JulienZLQ commented 2 months ago

@JulienZLQ I understand that, but the styles only add to the prompt. If I remember correctly, lllyasviel said somewhere that Fooocus should be able to handle long prompts. I could be wrong but I still believe it's an "issue" rather than user error.

This is how I understand it: too many conflicting prompts, combined with a certain degree of weight or parameter forced operation, will force the program to complete such conflicting synthesis operations, and the space for logical deduction will be severely compressed, resulting in the screen being forced to synthesize according to user requirements instead of being processed naturally. So I still think that too many conflicting prompts will cause errors due to excessive intervention, and it is not the ideal state of choosing and screening needs by itself.

mashb1t commented 2 months ago

@Catmechanic457 I don't see this as a bug but rather an user-introduced over-provisioning of style prompts. Please use prompt weight <= 2 (recommended max: 1.5) and do not use more than 3 styles to generate a good and guided result. If you need more specific styles feel free to write a preset for yourself or add a style to one of the existing style files.

lllyasviel / Fooocus