search

erkil1452 / gaze360

Code for the Gaze360: Physically Unconstrained Gaze Estimation in the Wild Dataset
http://gaze360.csail.mit.edu
Other
225 stars 42 forks source link

some problem about the predict #44

Closed jxncyym closed 2 years ago

jxncyym commented 2 years ago

@erkil1452 hello, I'm sorry to trouble you again. I test my own trained model using the camera that comes with the laptop, the camera is on the top-midde of the labtop. when I sit in the front of my labtop , the position of the eyes are on the same height of the camera, I got the correct gaze vector. However, when I test my own trained model using the video from the website, the gaze vector is not correct, I don't know why? whether I should do some thing like normalization of the head? another similar question is : when the tester is not in front of the camera, maybe there are some angles between the camera and the eyes(for example, the camera is on the left side or right side of the tester, or the camera looks up at the face), how can I get the correct gaze vector? thankyou!

liyixin-1 commented 2 years ago

@jxncyym I'm having the same problem as you, can you tell me about your reproduction process?

erkil1452 commented 2 years ago

The predicted gaze vector should be [0,0,1] when you look into the camera. It should not matter where the camera is.

jxncyym commented 2 years ago

@erkil1452 then what should I do to make the gaze vector to be [0,0,1] when look into the camera, could you describle the procedure detail? I found some describles in your paper image

for the formula image I want to confirm something :
1.the gL is the gaze vector predicted by our model ? and image is the norm of gL?

  1. then how to calculate the E when I predict the gaze using a video or a image capture with the camera within the labtop?
erkil1452 commented 2 years ago

You misunderstood. The output of the neural network should already be [0,0,1] (or -1?). You do not need to do anything.

jxncyym commented 2 years ago

@erkil1452 as you said, why I can't get the correct gaze vector when I predict the gaze vector with a video? but I can get the correct result when I use the camera within the labtop, as I sit in front of the labtop,and the eyes have the same height with the camera? I can't understand and I don't know why?

erkil1452 commented 2 years ago

So it works for you with images from a webcam? That is great! Then you just need to debug what is wrong with the video sequence. I am sure it must be some small error somewhere. Perhaps try to record video from your webcam to make the conditions as similar as possible.

jxncyym commented 2 years ago

ok, I will try it tomorrow, maybe there are still some question want to consult you

jxncyym commented 2 years ago

@erkil1452 I find the problem and solved it, thankyou

erkil1452 commented 2 years ago

Glad to hear that!