search

HARPLab / DReyeVR

VR driving 🚙 + eye tracking 👀 simulator based on CARLA for driving interaction research
https://arxiv.org/abs/2201.01931
MIT License
139 stars 37 forks source link

Register gaze at rear mirror #161

Open StijnOosterlinck opened 2 months ago

StijnOosterlinck commented 2 months ago

Hi,

In egosensor.cpp, the egovehicle is added as an ignoredactor for computing the gaze trace, which is useful. However, if I want to register the moments at which someone looks at the rear mirror, how could i do this? Essentially, I would want to add the vehicle as an ignored actor, except for the rear mirror, is this possible in some way?

Thx!

ajdroid commented 2 months ago

Do you need this in real time or is it okay for it to be in post-processing?

ajdroid commented 2 months ago

Off the top of my head, I think the easiest way to do this would be to replace the AddIgnoredActor with an AddIgnoredComponent and loop through all the components of the ego vehicle except the mirrors. You should be able to do Vehicle->GetComponents and then loop through them. Maybe log them to get an understanding of what all is in there

StijnOosterlinck commented 2 months ago

I don't need it in real-time, i'm only looking at the gaze intersections in post-processing. Does this change anything?

Else, I'll indeed look into adding ignored components instead of actors.

ajdroid commented 2 months ago

This may be more complicated but, in post-processing you can figure out what the region of 3D space is that corresponds to the mirror. This is constant in the ego vehicle's reference frame. Then, at each recorded timestep you can figure out if the gaze ray passed through that 3D space. Does that make sense?

StijnOosterlinck commented 2 months ago

Hi,

I tried the method of adding ignoredcomponents instead of the vehicle itselt. If i do this, then it seems like every gaze at the rear mirror is labeled as 'None'. I assume this happens because the rear mirror is part of the vehicle mesh, and the vehicle mesh is an ignored component.

I tried removing the vehicle mesh from the ignored components list, but then the gaze constantly intersects with that, so this is also not useful. How could i modify it such that the vehicle mesh is ignored, but the rear mirror not?

StijnOosterlinck commented 2 months ago

This may be more complicated but, in post-processing you can figure out what the region of 3D space is that corresponds to the mirror. This is constant in the ego vehicle's reference frame. Then, at each recorded timestep you can figure out if the gaze ray passed through that 3D space. Does that make sense?

If what I previously mentioned is not possible, could you elaborate on where and how I need to look to implement this?