Toggle pose for image

[Eagle07-Sudo]

In the past, I used to benefit from the feature of separating the main "InstantID" node from the "Face Keypoints Controlnet". So, sometimes I could generate image without a pose reference. However, after the recent modification, the pose has become mandatory, and it is not possible to provide an image without including the pose. So, I hope it is possible to have a way to turn off the pose, such as making the image_kps input mandatory for those who want to specify the pose, while the image input does not include the pose in any case.

[cubiq]

if you need to generate the image without controlnet it's better to use IPAdapter (any FaceID model).

You can send a black image in the image_kps if you really want but you are not taking advantage of InstantID.

[Eagle07-Sudo]

Ah, it's good to clarify that, thank you.

But, can you make load InstantID Model node load the file from the ComfyUI\models\ipadapter\instantid directory instead of ComfyUI\models\instantid? This way, it becomes visible to the Load IPAdapter Model node. Because currently, it is not possible to connect Load InstantID Model node to the Apply IPAdapter node.

[cubiq]

you can copy the model but that's not how they are meant to be used...

[Eagle07-Sudo]

Yes, I make a copy and I get this Error:

Error occurred when executing IPAdapterApply:

Error(s) in loading state_dict for Resampler: size mismatch for proj_in.weight: copying a param with shape torch.Size([1280, 512]) from checkpoint, the shape in current model is torch.Size([1280, 1280]).

[cubiq]

yeah like I said... they are not meant to be used that way

[Eagle07-Sudo]

Alright, I conducted tests and found it valuable to share a summary as it might be of interest to someone:

• It's evident that the best way to unlock the full features of InstantID is to use it conventionally and seamlessly connect its output. In fact, this is the only method to achieve the closest match to the facial features of the linked portrait image.
• The "IP-adapter" model does not work with recognized IPadapter nodes.
• The "Face Keypoint" controlnet model does not work with recognized Controlnet nodes.
• When using an image lacking a face in "image_kps," the face seems to lose its recognition of the portrait fed into the "image" input.
• Any image linked to the "image_kps" input will alter the result in some way, such as changing the seed, even if the image lacks a face. This influence is applied by the conditioning output only.
• The impact of connecting the model output alone while disconnecting the positive and negative conditioning outputs yields a futile result.
• Utilizing Controlnet "keypoints" alone can be beneficial by using the positive and negative outputs in the InstantID main node while disconnecting the model output. This method provides a boost with a subtle hint about the facial structure. The results are very interesting. And since the "Face Keypoints preprocessor" doesn't work alone, this is the only way Controlnet can be used, but the impact on the face in the final result is unavoidable.