metercai
commented
2 months ago Like #81 , bug caused by fooocus schema
Closed DavidDragonsage closed 2 months ago
metercai
commented
2 months ago Like #81 , bug caused by fooocus schema
DavidDragonsage
commented
2 months ago So it I understand correctly, this issue is resolved in the latest dev version - similar to #81 ?
metercai
commented
2 months ago The official version code has been updated and will be upgraded upon restart without the need to replace the program package. Can you verify if this bug still exists?
DavidDragonsage
commented
2 months ago Well we have partial success. The LoRAs do load correctly from the metadata, so that bug is fixed. Edited to add: WRONG! The LoRAs are not actually used but the logs indicate that they are in use.
However, SimpleSDXL is now creating images that are quite different from the originals. I have attached the new image created this morning, made from the metadata of the image I created yesterday. This image should be identical to the first image in this thread.
The image logs look identical, apart from the difference in how the LoRAs are listed, and the version number of course.
This is the metadata from the original image: { "prompt": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field", "negative_prompt": "blurry background, bokeh", "prompt_expansion": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field, dramatic light, cinematic, extremely detailed, beautiful, stunning, intricate, elegant, creative, surreal, highly advanced, thought, fancy, glorious, epic, clear, amazing, breathtaking, peaceful, vibrant, unique, artistic, incredible", "styles": "['Fooocus V2', 'Fooocus Semi Realistic', 'Dark Fantasy']", "performance": "Speed", "steps": 30, "resolution": "(896, 1152)", "guidance_scale": 7, "sharpness": 8, "adm_guidance": "(1.5, 0.8, 0.3)", "base_model": "SDXLFaetastic_v24.safetensors", "refiner_model": "None", "refiner_switch": 0.6, "clip_skip": 2, "sampler": "dpmpp_2m_sde_gpu", "scheduler": "karras", "vae": "Default (model)", "seed": "7841085968682328112", "lora_combined_1": "sd_xl_offset_example-lora_1.0.safetensors : 0.5", "lora_combined_2": "SDXLFaeTastic2400.safetensors : 0.7", "lora_combined_3": "blur_control_xl_v1.safetensors : 1.0", "lora_combined_4": "SDXL_black_and_color_Sa_May.safetensors : 1.0", "backend_engine": "SDXL-Fooocus", "metadata_scheme": "fooocus", "version": "Fooocus v2.5.5 SimpleSDXL_v20240916.2952150" }
This is the metadata from today's image: { "prompt": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field", "negative_prompt": "blurry background, bokeh", "prompt_expansion": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field, dramatic light, cinematic, extremely detailed, beautiful, stunning, intricate, elegant, creative, surreal, highly advanced, thought, fancy, glorious, epic, clear, amazing, breathtaking, peaceful, vibrant, unique, artistic, incredible", "styles": "['Fooocus V2', 'Fooocus Semi Realistic', 'Dark Fantasy']", "performance": "Speed", "steps": 30, "resolution": "(896, 1152)", "guidance_scale": 7, "sharpness": 8, "adm_guidance": "(1.5, 0.8, 0.3)", "base_model": "SDXLFaetastic_v24.safetensors", "refiner_model": "None", "refiner_switch": 0.6, "clip_skip": 2, "sampler": "dpmpp_2m_sde_gpu", "scheduler": "karras", "vae": "Default (model)", "seed": "7841085968682328112", "lora_combined_1": "sd_xl_offset_example-lora_1.0 : 0.5", "lora_combined_2": "SDXLFaeTastic2400 : 0.7", "lora_combined_3": "blur_control_xl_v1 : 1.0", "lora_combined_4": "SDXL_black_and_color_Sa_May : 1.0", "backend_engine": "SDXL-Fooocus", "metadata_scheme": "fooocus", "version": "Fooocus v2.5.5 SimpleSDXL_v20240916.b1b51ab" }
DavidDragonsage
commented
2 months ago I see what is happening. Although the LoRAs are listed in the metadata, they are having no effect.
I deselected all the LoRAs and regenerated the image. The program produced an image that is identical to the one above.
DavidDragonsage
commented
2 months ago From further experimentation I confirm that all LoRAs are inactive in Fooocus mode. Selecting a LoRA does not change the image at all, although the image log shows that the LoRA is in use.
In Comfy mode, loading metadata for a Flux image created with a LoRA creates an error - see the console log below. But Flux LoRAs do work if manually selected.
Loaded preset: F:\SimpleAI\SimpleSDXL2_win_0916\SimpleSDXL\presets\Flux🚀.json [Comfyd] Comfyd freeing! [ToolBox] Reset_params_from_image: -->Flux.1 params from the image with embedded parameters. [Fooocus] GPU memory: max_reserved=3.344GB, max_allocated=1.982GB, reserved=2.102GB, free=8.831GB, free_torch=0.120GB, free_total=8.951GB, gpu_total=12.000GB, torch_total=2.102GB [TaskEngine] Task_class:Flux, Task_name:Flux, Task_method:flux_base [TaskEngine] Enable Comfyd backend. [Comfyd] Comfyd is active! [Parameters] Adaptive CFG = 7 [Parameters] CLIP Skip = 2 [Parameters] Sharpness = 2 [Parameters] ControlNet Softness = 0.25 [Parameters] ADM Scale = 1.5 : 0.8 : 0.3 [Parameters] Seed = 5350975487230560396 [Parameters] CFG = 3.5 [Fooocus] Loading control models ... [Parameters] Sampler = euler - simple [Parameters] Steps = 12 - 30 [Fooocus] Initializing ... [Fooocus] Processing prompts ... [Wildcards] Copmile text in prompt to arrays: Detailed, masterpiece, professional, bold colors, awe-inspiring, photography inspired by modern fantasy art, medium shot, HDR, futuristic realistic geisha in a cyberpunk theme, neon geisha sparkle face paint, elaborate attire, street lights illuminate her face -> arrays:[], mult:0 [Parameters] Denoising Strength = 1.0 [Parameters] Initial Latent shape: Image Space (1152, 896) Preparation time: 0.01 seconds Using simple scheduler. [Fooocus] GPU memory: max_reserved=2.102GB, max_allocated=1.981GB, reserved=0.023GB, free=10.909GB, free_torch=0.015GB, free_total=10.924GB, gpu_total=12.000GB, torch_total=0.023GB [Fooocus] Preparing Flux task 1/1 ... [ComfyClient] Ready ComfyTask to process: workflow=flux_base2_gguf prompt = Detailed, masterpiece, professional, bold colors, awe-inspiring, photography inspired by modern fantasy art, medium shot, HDR, futuristic realistic geisha in a cyberpunk theme, neon geisha sparkle face paint, elaborate attire, street lights illuminate her face negative_prompt = width = 896 height = 1152 base_model = flux-hyp8-Q5_K_M.gguf sampler = euler scheduler = simple cfg_scale = 3.5 steps = 12 denoise = 1.0 seed = 5350975487230560396 clip_model = t5xxl_fp16.safetensors lora_1 = DracoFelis lora_1_strength = 1.0 [Comfyd] got prompt [Comfyd] Failed to validate prompt for output 27: [Comfyd] LoraLoaderModelOnly 31: [Comfyd] - Value not in list: lora_name: 'DracoFelis' not in (list of length 51) [Comfyd] Output will be ignored [Comfyd] invalid prompt: {'type': 'prompt_outputs_failed_validation', 'message': 'Prompt outputs failed validation', 'details': '', 'extra_info': {}} Traceback (most recent call last): File "F:\SimpleAI\SimpleSDXL2_win_0916\SimpleSDXL\modules\async_worker.py", line 1639, in worker handler(task) File "F:\SimpleAI\SimpleSDXL2_win_0916\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, *kwargs) File "F:\SimpleAI\SimpleSDXL2_win_0916\python_embeded\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) File "F:\SimpleAI\SimpleSDXL2_win_0916\SimpleSDXL\modules\async_worker.py", line 1445, in handler imgs, img_paths, current_progress = process_task(all_steps, async_task, callback_function, controlnet_canny_path, File "F:\SimpleAI\SimpleSDXL2_win_0916\SimpleSDXL\modules\async_worker.py", line 364, in process_task imgs = comfypipeline.process_flow(comfy_task.name, comfy_task.params, comfy_task.images, callback=callback) File "F:\SimpleAI\SimpleSDXL2_win_0916\python_embeded\lib\site-packages\simpleai_base\comfyclient_pipeline.py", line 154, in process_flow images = get_images(ws, prompt_str, callback=callback) File "F:\SimpleAI\SimpleSDXL2_win_0916\python_embeded\lib\site-packages\simpleai_base\comfyclient_pipeline.py", line 51, in get_images prompt_id = queue_prompt(prompt)['prompt_id'] KeyError: 'prompt_id' Total time: 0.91 seconds
DavidDragonsage
commented
2 months ago With today's auto-update from b1b51ab to fe3afb6 the LoRAs are again not loading from the metadata in Fooocus mode. However the Fooocus LoRAs do work if they are selected manually.
metercai
commented
2 months ago Your problem cannot be reproduced. Please simplify and follow the following process with attached images and logs: 1, Generate an image using default presets and an obvious lora, and select simple Metadata schema 2, Extract parameters from the previously generated image and generate another image 3, Cancel Lora and generate another image using the current parameters
DavidDragonsage
commented
2 months ago I understand what is happening now. I had assumed that the Simple designation was just a new name for Fooocus metadata. But I see now that Simple and Fooocus metadata are incompatible.
If I follow your directions - staying exclusively with Simple metadata - I get an exact replica.
But there is a more serious problem. I have a library of over 6000 Fooocus images on my hard drive. Should I choose to work on one of them in the SimpleSDXL win_906 version it is now impossible. And it is not unusual for me to rework old images as I gain more experience with AI generation.
I do not understand why the Fooocus metadata option was removed because this creates a significant incompatibility with mainline Fooocus. I had thought that Fooocus compatibility was one of the driving forces behind this project.
This is another reason I must keep the win_0820 version available on my SSD (the other reason being the better quality in Flux1 Dev). But that is not a good long term solution. And many people simply do not have the space to keep two or three variants of Fooocus on their drives (I did not until last week).
I have recently been testing Fooocus versus SimpleSDXL2. I was delighted to find that in both quality and speed they are a match (I am about to publish the results at the Pure Fooocus Facebook group). As far as quality goes, in some cases Fooocus will generate a better image with the same parameters and seed, and in other cases SimpleSDXL2 will generate a better image. And in many cases each image in the comparison was better in one respect but not as good in another.
My goal has been to actually eliminate both mainline Fooocus and SimpleSDXL win0820 from my PC. It appears that will not be practical.
metercai
commented
2 months ago The simple metadata schema is compatible with fooocus and can be read, but it will be converted to a new one when generated. This is because Simple needs to be compatible with other models, such as Flux and Kolors.
metercai
commented
2 months ago If there are any compatibility issues with reading of fooocus schema, you can submit bugs to fix them. When outputting, it must be a simple schema.
The a1111 schema can be output, but it is irreversible, meaning that it may not be restored after reading due to insufficient parameter information.
DavidDragonsage
commented
2 months ago Thank you, I am relieved that you intend to ensure that Fooocus metadata will load successfully into win2_960.
This thread has become somewhat confusing so I will start a new bug report focused on that one issue, of problems with loading Fooocus metadata.
Checklist
What happened?
When I wanted to work on an image I had created previously, I loaded the image into the metadata window and applied the metadata. Image creation appeared to proceed normally except that just before the image was saved this error occurred: "FileNotFoundError: [Errno 2] No such file or directory: 'F:\SimpleAI\SimpleModels\loras\sd_xl_offset_example-lora_1.0'" this error prevented the new image from being saved.
Steps to reproduce the problem
1) Create an image in the SimpleSDXL2 win2_960 version, using one or more LoRAs 2) Load and apply the metadata from that image 3) Regenerate the image 4) The image appears to regenerate properly but it does not save when the generation is complete. Instead an error appears in the console.
What should have happened?
The image should have regenerated properly.
What browsers do you use to access Fooocus?
Mozilla Firefox
Where are you running Fooocus?
Locally
What operating system are you using?
Windows 10
Console logs
Additional information
This problem only occurs when trying to regenerate an image from metadata that contains references to LoRAs.
The image will regenerate correctly if you manually select the LoRAs from the Models tab.
The image will regenerate correctly using the SimpleSDXL2 win2_820 version. No user intervention is required. This bug only affects the win2_960 version.
This is the image log file, and the attached image shows the generation when it works correctly.
{ "prompt": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field", "negative_prompt": "blurry background, bokeh", "prompt_expansion": "a Tamil man walks away and climbs the mystic never ending stairs on a dark and stormy night. Sharp background, deep depth of field, dramatic light, cinematic, extremely detailed, beautiful, stunning, intricate, elegant, creative, surreal, highly advanced, thought, fancy, glorious, epic, clear, amazing, breathtaking, peaceful, vibrant, unique, artistic, incredible", "styles": "['Fooocus V2', 'Fooocus Semi Realistic', 'Dark Fantasy']", "performance": "Speed", "steps": 30, "resolution": "(896, 1152)", "guidance_scale": 7, "sharpness": 8, "adm_guidance": "(1.5, 0.8, 0.3)", "base_model": "SDXLFaetastic_v24.safetensors", "refiner_model": "None", "refiner_switch": 0.6, "clip_skip": 2, "sampler": "dpmpp_2m_sde_gpu", "scheduler": "karras", "vae": "Default (model)", "seed": "7841085968682328112", "lora_combined_1": "sd_xl_offset_example-lora_1.0.safetensors : 0.5", "lora_combined_2": "SDXLFaeTastic2400.safetensors : 0.7", "lora_combined_3": "blur_control_xl_v1.safetensors : 1.0", "lora_combined_4": "SDXL_black_and_color_Sa_May.safetensors : 1.0", "backend_engine": "SDXL-Fooocus", "metadata_scheme": "fooocus", "version": "Fooocus v2.5.5 SimpleSDXL_v20240731.baa9f01" }