Illusion model not loading

[djooodj]

Since the controlNet 3d to image workflow was updated, I've not been able to successfully use the Illusion model. I don't get any errors, but its just returning results that stylize the input strip. The original image is clearly featured, not hidden as expected.

Strip input processing started (ctrl+c to cancel). Use file seed and prompt: Yes Total VRAM: 17170825216 Total GPU Cards: 1

1/1 Prompt: cinematic film still hatch_txt, a mossy forest floor with twigs and forest undergrowth, sunset lighting, high angle wide shot. shallow depth of field, vignette, highly detailed, high budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy Negative Prompt: anime, cartoon, graphic, text, painting, crayon, graphite, abstract, glitch, deformed, mutated, ugly, disfigured Load: ControlNet Model unet\diffusion_pytorch_model.safetensors not found Loading pipeline components...: 0%| | 0/7 [00:00<?, ?it/s]text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["id2label"] will be overriden. text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["bos_token_id"] will be overriden. text_config_dict is provided which will be used to initialize CLIPTextConfig. The value text_config["eos_token_id"] will be overriden. Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 7/7 [00:03<00:00, 2.00it/s] Load Refine Model: stabilityai/stable-diffusion-xl-refiner-1.0 Loading pipeline components...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:01<00:00, 4.82it/s] Seed: -1078186375 Process: ControlNet Token indices sequence length is longer than the specified maximum sequence length for this model (96 > 77). Running this sequence through the model will result in indexing errors The following part of your input was truncated because CLIP can only handle sequences up to 77 tokens: ['budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy'] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:03<00:00, 8.31it/s] Refine: Image Token indices sequence length is longer than the specified maximum sequence length for this model (96 > 77). Running this sequence through the model will result in indexing errors The following part of your input was truncated because CLIP can only handle sequences up to 77 tokens: ['budget, bokeh, cinemascope, moody, epic, gorgeous, film grain, grainy'] 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 12/12 [00:01<00:00, 8.79it/s] Warning: 1 x Draw window and swap: 103.0201 ms, average: 103.02010000 ms Processing finished.

these are my settings:

[tin2tin]

Try quality steps between 2 and 3.

[djooodj]

Thanks @tin2tin , it seems like it is indeed working. I feel like I'm not learning exactly the fix though. I tried lowering the quality steps as suggested, and I was able to immediately see that it is referencing my input strip text. Increasing the quality steps from there did not undo this progress, which surprised me. I'm now back to 25 on Quality Steps and the renders are working more as expected, so that part is confusing. I've played with word power, though now I'm back to the default 9 and it's working fine. I have played with the seed as well, but no clear differences there.

I've found (I think) that "Refine image" possibly obscures the illusion too far? But honestly, it's not so much that I'd feel as if the input image was ignored, so I don't think that explains my previous issue either.

I would also say that a particular prompt I was trying before was just very unsuccessful. It was something like: Aerial shot of a forest in Autumn with a single gravel road winding through. I never got a result that seemed to include the input strip with the Illusion model, no matter the quality level changes.

Would it be possible to add hover tool tips about each parameter to better understand the users decisions? Thank you so much, again.

[tin2tin]

Tooltips would mean that I would have to understand what effect each parameter has, and to me it seems like a very "dynamic" system, where nothing can be set in stone, exactly as you experience. If you're able to narrow the functions down to something explaineable, please share.

[tin2tin]

I hope this can be considered solved.

[djooodj]

Yes, I think I don't have the experience to add to the description of those functions, so I guess the issue should be closed! Thank you.