Change the base model

[epicstar7]

Amazing extension for out painting, thanks for that!

I was wondering what the base SDXL model actually does to the image in this case. The RealVisXL_V5.0_Lightning_fp16 is recommended and I put it in the folder. Then I thought let's try other models, so I placed other models in the folder but same results. Then I completely removed the model from the folder just to check, but the extension still generates images inside comfy. I checked the code but couldn't really find clues what it's really doing for the image generation.

Does this model even do anything? And if so, how do I actually change it to another model?

Thanks!

[GiusTex]

Amazing extension for out painting, thanks for that!

Very happy to hear that!

I was wondering what the base SDXL model actually does to the image

Controlnet is passed to the model, and toghether they form the generation pipe.

Then I completely removed the model from the folder just to check, but the extension still generates images

The extension should unload the model after each generation, but maybe it saved it, did you try restarting comfyui without the model?

how do I actually change it to another model?

• Thanks for asking, cause I noticed the extension is actually forced to use 2 tokenizers and 2 text_encoders (which is default for sdxl and flux, I think, but not for sd1.5 for example), I'll change it today (to use what's in the model folder). For now all sdxl and ~flux~ models should work, but I still have to test it (probably flux won't be so good with its current controlnet model).
• After I'll have updated utils.py, sd1.5 models should work too (having 1 tokenizer and text_encoder, not 2). Sd1.5 + 1.5 controlnet models, of course.
• (Also, if I could load a civitai model and pass the model output of the comfyui simple model loader node, and maybe the clip output too (I tried the clip but didn't found a way to make it work), we could skip the diffusion model loader node... new tons of tests and searches to do, yuppi).

How to change diffuser model: You can go to huggingface and search for text2image models, search link, then choose a model and create a new folder named after it in comfyui/models/diffusion_models, then download in it the subfolders unet, tokenizer, text_encoder (and tokenizer_2 and text_encoder_2 it they exists). I should probably write this in the readme. Then in comfyui choose the new model. This is all theory, sorry, I'm on the phone outside but today I should be able to make some tests, and anyway this is the path I'll follow

[GiusTex]

how do I actually change it to another model?

I'm making some updates on the test branch. Now the model (unet) and controlnet are left, then changing model will be, easier, quicker (and available)

[epicstar7]

Thanks so much @GiusTex, can I already test this? I am not sure how this works with installing it on comfy from a test branch.

[GiusTex]

Yeah, you can test it, as I wrote in the updates part, typing git checkout Test, then git pull, I tested it in comfyui and only when it worked I created the new branch; you can also go back to main branch later. I updated the workflows too (checkpoint loader simple)

[epicstar7]

I am getting this:

_` F:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-DiffusersImageOutpaint>git checkout Test Switched to branch Test

F:\pinokio\api\comfy.git\app\custom_nodes\ComfyUI-DiffusersImageOutpaint>git pull There is no tracking information for the current branch. Please specify which branch you want to merge with. See git-pull(1) for details.

git pull <remote> <branch>

If you wish to set tracking information for this branch you can do so with:

git branch --set-upstream-to=origin/<branch> Test

`_

[GiusTex]

Please specify which branch you want to merge with

Try git pull ComfyUI-DiffusersImageOutpaint Test

[GiusTex]

I updated the info. Now for the main model just the unet and scheduler are required, and same for the controlnet (except it doesn't need the scheduler)