Different results on different machine

[AnkS4]

While this may be slightly unrelated, I couldn't find out why the deepface models are returning different result for same code:

from deepface import DeepFace
from sklearn.datasets import fetch_lfw_pairs
from sklearn.metrics import accuracy_score

X = fetch_lfw_pairs(
    subset="test",
    funneled=False,
    # slice_=(slice(0, 250), slice(0, 250)),
    resize=1,
    color=True
  )

imgs, y = X.pairs, X.target

preds = []

for i in range(len(imgs)):
  img = imgs[i]

  img1 = img[0]
  img2 = img[1]
  img1 = img1[:,:,::-1]
  img2 = img2[:,:,::-1]

  result = DeepFace.verify(
      img1,
      img2,
      model_name="ArcFace",  # "DeepFace" # "ArcFace" # "Dlib" # "DeepID" # "OpenFace" # VGG-Face" # "Facenet"
      detector_backend="dlib",  # "mtcnn",  #"ssd" # "opencv",  # "retinaface",
      enforce_detection=False,
      )

  # print(f"Actual :", y[i], "Predicted :", result["verified"])
  preds.append(result["verified"])  # result

print("Accuracy :", 100*accuracy_score(y, preds))

Machine 1 (i.e. Google Colab) is returning 94% accuracy. Machine 2 is returning 50% accuracy.

The library versions on two different platform is as follows:

             Machine 1                     Machine 2
deepface     0.0.75                        0.0.75
scikit-learn 1.0.2                         1.1.1
opencv       4.6.0.66                      4.6.0.66
tensorflow   2.8.2+zzzcolab20220527125636  2.7.0

Could scikit-learn version be the cause?

[serengil]

most probably it is because of the weight file in HOME/.deepface/weights. Delete them and let the library to re-install.

[AnkS4]

I retested after deleting the weights, but the results staying the same.

md5sum of weight files are same on both: dfe5b4da0a49aef19ec4c62673295d29 arcface_weights.h5

[serengil]

it seems your environment is problematic. because you should have same results with same input and weight pair. would you please reinstall tensorflow?

btw, what are your python and tf versions?

[chandniagarwal]

Please add

!pip install mediapipe

[AnkS4]

Most likely the issue with scikit-learn fetch_lfw_pairs function:

https://github.com/scikit-learn/scikit-learn/issues/23750