Face indexes messed up by gender detection

[darkpixel]

First, confirm

• [X] I have read the instruction carefully
• [X] I have searched the existing issues
• [X] I have updated the extension to the latest version

What happened?

Create two "collages" of faces. One of your female friends (let's say there are 5 of them in the image), and the other of your male friends (say there are 5 of them in the image).

Use the image loader to load them. Create two face swap nodes. Set one to detect female input, and the other to detect male input. Set some sort of source and input index. i.e. 0,1,2,3,4 and 0,1,2,3,4

Generate an image and pass it through the female input and connect the female output to the male input. Connect the male output to a "save" node.

Now generate an image....for example "5 guys hanging out on a beach". All five faces will be replaced in order (assuming gender detection does its job correctly). i.e. the source image face 0 will replace the input image face 0, and so on.

Now generate a mix of genders. i.e. "Two guys and two girls playing pool at a bar".

Let's say the generate image (from left to right) goes: girl, boy, girl, boy

What I would expect given the source and input indexes set to 0,1,2,3,4 is that the first two women in the source image and the first two men in the source image are used. But that's not what happens. From the women, women 0 and 2 are applied, and men 1, and 3 are applied.

It looks like the indexes are applied before gender detection.

I think the indexes should be applied after gender detection.

Steps to reproduce the problem

Your workflow bugreport.json

[darkpixel]

I guess I should add, the reason this makes things difficult is if you have a male and female facemap loaded, i.e. my wife is female 0 and I am male 0, and I generate a bunch of us at a bar playing pool, it's not really that important who gets what face...but if I decide I want to make an image of just the two of us playing pool, I will get semi-random faces depending on if the male is on the left or right.

If the female is on the left, the female face will be female 0 (my wife), and the male face will be male 1 (one of my friends, not me). If the male is on the right, the male face will be male 0 (me), and the female face will be female 1 (one of my friends, not my wife).

If you're using faceswap to generate scenes with a primary male and female character and the rest of the faces in the facemaps are for when you occasionally have "extra characters" in the scene, it won't work--meaning you need to adjust your facemap indexes for every image generated.

[Gourieff]

You can try to set indexes greater than you have faces in the scene E.g. you have two person in almost all the images but sometimes you have 3 - so you can set 0,1,2 for all the frames you have This index will work for both 2 and 3 faces

But yes, faces can be swapped wrong when some extra person appears in the middle I will add a new option soon which will allow to swap not by "left to right" but by "big to small"

[darkpixel]

Yup. I already have a "facemap" image of male friends, and another one of female friends.

The first image in my male facemap is me, the first image in my female facemap is my wife, and I set the index to 0,1,2,3,4,5 for both of them so as more "friends" get added to the picture it has them available for swapping.

I'm curious what you mean by "big to small". Are you talking about something that's sort of a "depth of field" from the camera? So close-up faces start with 0 and increase farther away?

[Gourieff]

I'm curious what you mean by "big to small". Are you talking about something that's sort of a "depth of field" from the camera? So close-up faces start with 0 and increase farther away?

Yes, smth like this Big faces go first and than smaller This will help to avoid any misswapping in crowded images where the "main face" is not the first in a row (if we count from left to right and up-down)

[darkpixel]

Interesting idea.

[darkpixel]

So I just played with the latest release, and there are a lot of great new features--including the "large to small" face ordering.

Unfortunately gender detection still messes up index numbers.

If I set it to use "large to small" and set gender detection to "male", face 0 might still be a female face....reactor just won't replace it because it's the wrong gender.

It would be much easier to target replacements if the index numbers were generated after gender detection.