Analyzer does not detect any faces when using UniversalReader

[timlac]

I have tried the demo and everything works fine as long as I use the path_image parameter and pass an image path to the analyzer.run function. However, if I feed the analyzer.run with an image converted to numpy array instead no faces are detected at all.

Adapting code in the demo to make a minimal reproducible example of the issue:

import cv2
from facetorch import FaceAnalyzer
from omegaconf import OmegaConf
import os

os.environ["HYDRA_FULL_ERROR"] = "1"

path_img_input = "./test.jpg"
path_config = "gpu.config.yml"

cfg = OmegaConf.load(path_config)

# Modify the configuration to include only the 'fer' and 'va' predictors
cfg.analyzer.predictor = {"fer": cfg.analyzer.predictor.fer, "va": cfg.analyzer.predictor.va}

# Remove all utilizers from the configuration
cfg.analyzer.utilizer = {}

analyzer = FaceAnalyzer(cfg.analyzer)

# Load the image using OpenCV
image = cv2.imread(path_img_input)

# Convert the image to RGB format (OpenCV loads images in BGR format)
image_rgb = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)

# Pass the NumPy array to the analyzer
response = analyzer.run(
    image_source=image_rgb,
    batch_size=cfg.batch_size,
    fix_img_size=cfg.fix_img_size,
    include_tensors=cfg.include_tensors,
)

The code above will not result in any detections, while if I run the exact same code but with the path_image parameter (as in the demo), 4 faces are detected.

As instructed here I have replaced ImageReader with UniversalReader in my gpu.config.yml file.

analyzer:
  device: cuda
  optimize_transforms: true
  reader:
    _target_: facetorch.analyzer.reader.UniversalReader
    device:
      _target_: torch.device
      type: ${analyzer.device}
    optimize_transform: ${analyzer.optimize_transforms}
    transform:
      _target_: torchvision.transforms.Compose
      transforms:
      - _target_: facetorch.transforms.SquarePad
      - _target_: torchvision.transforms.Resize        
        size:
        - 1080
        antialias: True
...

Using facetorch version 0.5.0

[tomas-gajarsky]

Thank you for reporting this issue and providing a detailed example. The problem you're encountering is due to how UniversalReader handles NumPy arrays in version 0.5.0. Specifically, the old implementation assumes a certain format when converting the NumPy array to a PIL image, which can lead to incorrect processing and no face detections.

In version 0.5.1, we've updated the UniversalReader to handle NumPy arrays more robustly:

• Old behavior: Converts the NumPy array to a PIL image, which might not preserve the correct image format.
• New behavior: Directly converts the NumPy array to a PyTorch tensor, ensuring the dimensions and channels are correctly interpreted.

This change should resolve the issue you're experiencing when passing a NumPy array to analyzer.run.

Please update to facetorch version 0.5.1 and try running your code again. It should now correctly detect faces from the NumPy array input.

Let me know if you have any further questions or run into other issues!