Got a bad face on some models

[TheHonestBob]

I am replacing different models for testing and use ip-adapter-plus-face_sdxl_vit-h.bin and ip-adapter-plus-face_sd15.bin, But in most of the tested models, bad faces appeared the model is https://civitai.com/models/102893/helloflatanime my code is pipe = StableDiffusionPipeline.from_single_file( sd_path, original_config_file=r'config/v1-inference.yaml', torch_dtype=torch.float16, scheduler=scheduler, vae=vae, local_files_only=True, feature_extractor=None, load_safety_checker=None ) ip_ckpt = os.path.join(ip_dir, "models/ip-adapter-plus-face_sd15.bin") ip_model = IPAdapterPlus(pipe, image_encoder_path, ip_ckpt, device, num_tokens=16) What should I do, or if there are any errors in my code, I haven't found the answer in previous issuses

[TheHonestBob]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

[xiaohu2015]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

[TheHonestBob]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

I tried demo cases，I also try stable-diffusion-xl-base-1.0、stable-diffusion-v1-5 and other a few base model on civitai，such as https://civitai.com/models/142320?modelVersionId=192793, The same code, these models are normal, but I have this problem when use other model downloading from civitai. However, when I try to reduce the scale parameter, such as 0.05 or 0.01, it is normal, but the corresponding faces are basically different, such as https://civitai.com/models/102893/helloflatanime, Can reproduce this problem

[xiaohu2015]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

I tried demo cases，I also try stable-diffusion-xl-base-1.0、stable-diffusion-v1-5 and other a few base model on civitai，such as https://civitai.com/models/142320?modelVersionId=192793, The same code, these models are normal, but I have this problem when use other model downloading from civitai. However, when I try to reduce the scale parameter, such as 0.05 or 0.01, it is normal, but the corresponding faces are basically different, such as https://civitai.com/models/102893/helloflatanime, Can reproduce this problem

so only some models can work well? in fact, ip-adapter should have a good generalization

[TheHonestBob]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

I tried demo cases，I also try stable-diffusion-xl-base-1.0、stable-diffusion-v1-5 and other a few base model on civitai，such as https://civitai.com/models/142320?modelVersionId=192793, The same code, these models are normal, but I have this problem when use other model downloading from civitai. However, when I try to reduce the scale parameter, such as 0.05 or 0.01, it is normal, but the corresponding faces are basically different, such as https://civitai.com/models/102893/helloflatanime, Can reproduce this problem

so only some models can work well? in fact, ip-adapter should have a good generalization

Yes, most of my tests have bad faces, so how can I solve this problem? It shouldn't be my code problem. Reducing the scale will prevent bad faces, but the faces are not similar anymore，In addition, regarding the issue of loading Lora, we look forward to resolving it

[xiaohu2015]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

I tried demo cases，I also try stable-diffusion-xl-base-1.0、stable-diffusion-v1-5 and other a few base model on civitai，such as https://civitai.com/models/142320?modelVersionId=192793, The same code, these models are normal, but I have this problem when use other model downloading from civitai. However, when I try to reduce the scale parameter, such as 0.05 or 0.01, it is normal, but the corresponding faces are basically different, such as https://civitai.com/models/102893/helloflatanime, Can reproduce this problem

so only some models can work well? in fact, ip-adapter should have a good generalization

Yes, most of my tests have bad faces, so how can I solve this problem? It shouldn't be my code problem. Reducing the scale will prevent bad faces, but the faces are not similar anymore，In addition, regarding the issue of loading Lora, we look forward to resolving it

do you tested on webui or comfyui?

[TheHonestBob]

Perhaps I have found the reason, I should try lower scale parameters, such as 0.1, 0.05，May I ask if this method is correct

have you tested the demo cases?

I tried demo cases，I also try stable-diffusion-xl-base-1.0、stable-diffusion-v1-5 and other a few base model on civitai，such as https://civitai.com/models/142320?modelVersionId=192793, The same code, these models are normal, but I have this problem when use other model downloading from civitai. However, when I try to reduce the scale parameter, such as 0.05 or 0.01, it is normal, but the corresponding faces are basically different, such as https://civitai.com/models/102893/helloflatanime, Can reproduce this problem

so only some models can work well? in fact, ip-adapter should have a good generalization

Yes, most of my tests have bad faces, so how can I solve this problem? It shouldn't be my code problem. Reducing the scale will prevent bad faces, but the faces are not similar anymore，In addition, regarding the issue of loading Lora, we look forward to resolving it

do you tested on webui or comfyui?

I will try later,thanks for your reply