search

hysts / pytorch_mpiigaze_demo

Gaze estimation using MPIIGaze and MPIIFaceGaze
MIT License
300 stars 68 forks source link

Performance of the model trained on MPIIFaceGaze #23

Closed Zagreus98 closed 2 years ago

Zagreus98 commented 2 years ago

Hi @hysts ,

First of all thank you for your great work, I learned a lot from it and it was really useful. I want to perform some experiments with the model which is trained on the MPIIFaceGaze dataset and I was just wondering if you have the perfomance ( like val angle error) for the model with the pretrained backbone resnet18 available here, it would be nice if I can compare the results, if you remember the hyperparameters setting for this one it would be even better.

LATER EDIT: Just realised is called resnet14 because layer4 is deleted. Thus, my guess is the mean angle error of the pretrained MPIIFaceGaze model is the one presented here and is 4.83 deg am I right?

hysts commented 2 years ago

Hi, @Zagreus98

LATER EDIT: Just realised is called resnet14 because layer4 is deleted. Thus, my guess is the mean angle error of the pretrained MPIIFaceGaze model is the one presented here and is 4.83 deg am I right?

Yes, that's the case. If I remember corerctly, the config file in the repo was the one I used to train the pretrained models I provided. I trained with cudnn non-deteministic setting, so the result varies from run to run and it's not possible to reproduce completely, but I think you can get similar result with the config.

Zagreus98 commented 2 years ago

Thanks for the reply. Indeed, I got similar results by training with the settings already present into the config. Just to be sure, because I'm not sure if I understood this graph correctly. From my understanding, there are 15 different models ( each one being trained on all the persons data, except person0n , n being the number of the current experiment) which are tested on different person data. Or there is a single model tested on each person data individually? image

hysts commented 2 years ago

@Zagreus98

From my understanding, there are 15 different models ( each one being trained on all the persons data, except person0n , n being the number of the current experiment) which are tested on different person data.

Yes. And the the pretrained model provided was trained on all data of 15 individuals with the same hyperparameters as the experiment of the graph.

Zagreus98 commented 2 years ago

Nice, thanks for the info! It's clear now :)