How to reproduce the result of verification from the analyzer using only the predictor from it?

[teodred]

Hi! First of all, thank you for your work, it looks great. I encountered some problems with reproducing results of face verification using it out of the pipeline that is implemented in the analyzer class. The reason why I want to use only the predictor but not the whole pipeline is that I have already implemented the face detector and I want to verify faces between each other, so I don't need most of the functionality that is present in the analyzer. And the problem is that scores of cosine similarity of output from the analyzer and single predictor are different, and it breaks the verification. Here is the code that I use:

` test_img = cv2.imread('./database/17/17_01-00-399.jpg')

test_img_2 = cv2.imread('./database/13/13_00-09-199.jpg')
# #13/13_00-09-199.jpg

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

path_img_input='./database/17/17_01-00-399.jpg'
path_img_input_2 = './database/13/13_00-09-199.jpg'
path_img_output="/test_output.jpg"
path_config="gpu.config.yml"

test_img = np.transpose(test_img, (2,0,1))
test_img = np.expand_dims(test_img,axis=0)
test_torch = torch.Tensor(test_img)
test_torch = test_torch.to(device=device, dtype=torch.float32)
test_torch_r = test_torch
test_torch = test_torch/255

test_img_2 = np.transpose(test_img_2,(2,0,1))
test_img_2 = np.expand_dims(test_img_2, axis=0)
test_torch_2 = torch.Tensor(test_img_2)
test_torch_2 = test_torch_2.to(device=device, dtype=torch.float32)
test_torch_2 = test_torch_2/255

cfg = OmegaConf.load(path_config)

predictor = instantiate(cfg['analyzer']['predictor']['verify'])
analyzer = FaceAnalyzer(cfg.analyzer)

responce_a = analyzer.run(path_image=path_img_input,
    batch_size=cfg.batch_size,
    fix_img_size=cfg.fix_img_size,
    return_img_data=True,
    include_tensors=True,
    path_output=path_img_output,)
responce_a2 = analyzer.run(path_image=path_img_input_2,
    batch_size=cfg.batch_size,
    fix_img_size=cfg.fix_img_size,
    return_img_data=True,
    include_tensors=True,
    path_output=path_img_output,)

responce = predictor.run(test_torch)[0].logits
responce_2 = predictor.run(test_torch_2)[0].logits

vec_1  = responce_a.faces[0].preds['verify'].logits
vec_2 = responce_a2.faces[0].preds['verify'].logits

sim = cosine_similarity(vec_1, vec_2, dim=0)
sim_3 = cosine_similarity(responce, responce_2, dim=0)

print(sim)
print(sim_3)

` And as result i get tensor(0.3768, device='cuda:0') tensor(0.0573, device='cuda:0') So predictor by itself doesn't work. Thank you in advance.

[tomas-gajarsky]

Hi, I had a quick look at the code you provided and I think that the problem might be in the order of colors. Cv2 imread returns a BGR image array while the face verification model works with RGB. Try to change the order of channels in your test_torch tensors using the rgb2bgr function from facetorch.utils module (it also works for bgr2rgb) and let me know if that provides the correct results.

[teodred]

Thank you for your response. I have tried to change the order of channels. But it didn't help. Here is the new results:

---

tensor(0.1691) result of analyzer tensor(0.6214) result of predictor

---

Result of analyzer is different, because i have picked different photos.

[tomas-gajarsky]

Alright, thanks for trying to change the order of colors. I have some additional questions that could help me say what the issue is:

Which result makes more sense to you? Are the faces from the two images one person or different people?

Do the images contain single cropped-out faces? What's the resolution of these images?

[tomas-gajarsky]

FaceAnalyzer resizes the input images to 1080 x 1080 resolution using the ImageReader. The results might be different if your input image is much smaller, which might be true for cropped-out faces.

Change the size of the ImageReader Resize transform in /conf/analyzer/reader/default.yaml to the length of the longer side of the specific input image.

[wangting0]

path_config="gpu.config.yml"

Hello, how did you write this config file? Could you please tell me about it? Thank you very much！

[tomas-gajarsky]

path_config="gpu.config.yml"

Hello, how did you write this config file? Could you please tell me about it? Thank you very much！

You can use the merged config file or its GPU version located in the same directory.