search

hysts / pytorch_mpiigaze_demo

Gaze estimation using MPIIGaze and MPIIFaceGaze
MIT License
290 stars 66 forks source link

Which coordinate system? #11

Closed TNA8 closed 2 years ago

TNA8 commented 2 years ago

I am trying to get on-screen 2d positions from 3d gaze vectors calculated mpiigaze. I used mirror-based external camera calibration. The 2d gaze vectors are very not accurate.

Thanks in advance.

hysts commented 2 years ago

Hi, @TiennAhn

I think gaze vectors are computed in the camera coordinate system.

TNA8 commented 2 years ago

The output gaze vector is in original camera coordinate system or normalized camera coordinate system? I am still confusing. If I need gaze vector in original camera coordinate system, which method can we use? screenshot_19 screenshot_20

Could you make it clear?

TNA8 commented 2 years ago

Can I directly use the calculated 3d gaze vector in calculating on-screen 2d gaze points?

hysts commented 2 years ago

@TNA8 The raw output of the MPIIGaze model is in the normalized coordinate system, but it's post-processed to be in the original camera coordinate system.

See the following: https://github.com/hysts/pytorch_mpiigaze_demo/blob/master/ptgaze/gaze_estimator.py#L97 https://github.com/hysts/pytorch_mpiigaze_demo/blob/master/ptgaze/common/face_parts.py#L39-L45

TNA8 commented 2 years ago

screenshot_21 screenshot_22

This is the real output values. The values is very small, seems not in original camera system.

hysts commented 2 years ago

@TNA8

The length of gaze vector is meaningless here. It's not that the end point of the gaze vector is the 3D point the person is looking at. It just means the person is looking at the direction the vector is pointing.

TNA8 commented 2 years ago

Yes, but the eye.center value is almost (0, 0, 0). This could not be in original camera system.

hysts commented 2 years ago

@TNA8

Ah, OK. As you underlined the origin and the end point, I thought you were talking about the length of the vector, but it wasn't.

You say the coordinate values are small, but it depends on which metric system is used, right? Here, the unit is meters (see this), so it actually looks like a decent value to me, though I'm not sure because the expected values depend on the input video you used.

TNA8 commented 2 years ago

So the final 3d gaze vector from the model is in original camera system and the unit is meter. Right?

hysts commented 2 years ago

Yes.

TNA8 commented 2 years ago

Thanks a lot for your quick and kind explanation.

younesmch commented 2 years ago

@TNA8 i have the same problem as you can you help if you done for getting gaze on screen how you solve the problem thnks