Closed Starzilla29 closed 6 months ago
I'm having the same issue - the second instantid+controlnet nodes (of the second face) in the conditioning chain override the first instantid+controlnet nodes (of the first face). The result is that both controlnet nodes that control the location of the faces end up generating the same face.
the workflow presented by OP, has various issues. The comfyui controlnet node doesn't have a mask function so there will be some bleeding, there are some workarounds, we'll discuss this in the future but for now let's leave this as "only partially supported"
It's complicated but possible. I've updated the multi-id workflow in the repository. The workflow can be further improved by giving a better context to the whole composition (with an additional prompt or controlnet)
Hi @cubiq , I reviewed the official code and did some experiments. I think the actual procedure is: Given two reference image (providing ID),
I don't think that masking the attention is enough because the embeds are also in the controlnet, anyway I have an idea to simplify the workflow. I'll work on it when I have time
I don't think that masking the attention is enough because the embeds are also in the controlnet, anyway I have an idea to simplify the workflow. I'll work on it when I have time
Do you have some insight for doing that in a image2image fashion (style transfert) ? My current approach is to detect all the face in the image and make a mask using GroundingDINO of SegCLIP. And than apply InstantID with attention masking on each corresponding mask
I have been messing with the Multi-ID workflow provided in this repo. I wanted to try it with a separate man and woman that had very different features from each other, the idea being I wanted to make sure I could use InstantID with multiple faces in the same images.
I modified the repo to have a fix seed and generate all images including the mask (for ease of testing instead of having to load a bunch of images from disk). I generated an image of an old man and a young adult blonde woman. I have also generated a face for face keypoints to use as a pose. Ideally I get the older man on the left side of the image and the woman on the right. however, using this workflow, with both apply instantID nodes I only get the woman on both sides. Even if I turn the weight all the way down for the woman, I still get the woman and never the man. However, removing the woman's apply instantID and respective controlnet allows the man to come through on just the left side as he is supposed to. Below is the workflow I am using: Multi-ID-Testing-Workflow.json
The three generated references images from the workflow to be used as inputs:
![ComfyUI_temp_hirty_00004_](https://github.com/cubiq/ComfyUI_InstantID/assets/47699576/1c8b2d05-4326-4a87-84f3-dbba13ef0ab4)
The results of the workflow:
![ComfyUI_temp_lfnya_00032_](https://github.com/cubiq/ComfyUI_InstantID/assets/47699576/6195f256-ca17-440b-8d38-e5a1f8ddbd67)
Based on the behavior I am seeing, it looks like the last apply InstantID is completely overriding the first. I believe this to be a bug, unless there is something completely wrong that I missed that I am supposed to be doing that I am not. However, this is basically the workflow provided with the exception that the images and mask have been changed to be generated from the network and a resolution change.