Where can i get the eyecar video data

[Taoismer]

Where can i get the eyecar video data?

[soniabaee]

Hi, it depends on what the part of the video:

• you can find the raw data here Videos
• the frame here Frames
• and fixation data here Fixation

[Taoismer]

Thx. I cannot understand. Should it have some video data like *.mp4. I can not see any this kind of file in the EyeCar dir.

[soniabaee]

Unfortunately, the videos are not publicly disclosed as it's mentioned in the paper. However, all of the information about the scene of each videos as well as drivers' eye movement are available in this repo.

To create EyeCar we used the Naturalistic Driving Study (NDS) for the second Strategic Highway Research Program (SHRP 2) videos and capture the eye movement of drivers. You can find more information about EyeCar in the appendix of our paper here. For your reference, the details about the SHRP 2 data is available in this file. You also can register to download the videos here.

[Taoismer]

Unfortunately, the videos are not publicly disclosed as it's mentioned in the paper. However, all of the information about the scene of each videos as well as drivers' eye movement are available in this repo.

To create EyeCar we used the Naturalistic Driving Study (NDS) for the second Strategic Highway Research Program (SHRP 2) videos and capture the eye movement of drivers. You can find more information about EyeCar in the appendix of our paper here. For your reference, the details about the SHRP 2 data is available in this file. You also can register to download the videos here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[soniabaee]

I cannot quite understand your question, but please let me know which classes are missing. I'll be more than happy to help.

[Taoismer]

for example, in the main.py, I cannot find the code of the visual and driving module defined in any other files.

[soniabaee]

The version that I've uploaded doesn't have all the functions. This is on-going project and I will add the cleaned version of the code ASAP. Thanks for letting me know. I also added the to-do list in the readMe to address this issue.

[Taoismer]

Thanks for your work. One more question: can you share the dataset of EyeCar with video data with me? It is hard for me to access the right video data from SHRP2.

[Taoismer]

Hi, I wonder how the gridworld is modeled your work. The number of the grid cell for each image seems to be a variable, and the same as the fixation points. How do you opreate these fixation points with 7 actions in the paper？

[soniabaee]

Hi, sure! the number of the grid as you mentioned is a parameter but for each video (21 videos), not each frame. We determine it based on the smallest(furthest) size of the lead-vehicle in the corresponding video (12 X 17) is the smallest lead-vehicle in all of our videos. We averaged and smoothed the eye fixations of all of our drivers (20 participants) per frame so each frame of a video can have multiple fixations.

After discretizing the frame and knowing the fixations in each frame. The policy selects one patch (one cell) as the next fixation location. We calculate the 2-D changes of the new fixation location and the current fixation location to have 7 actions (up, down, left, right, …). Please check out the appendix of the paper for more detail.

[Taoismer]

Yes. But I wonder what does the policy output? the next patch? or one of the seven actions? If the latter, how can we control several fixations over one frame with one action(up,down,left,right...).THX.

---

Hi, sure! the number of the grid as you mentioned is a parameter but for each video (21 videos), not each frame. We determine it based on the smallest(furthest) size of the lead-vehicle in the corresponding video (12 X 17) is the smallest lead-vehicle in all of our videos. We averaged and smoothed the eye fixations of all of our drivers (20 participants) per frame so each frame of a video can have multiple fixations.

After discretizing the frame and knowing the fixations in each frame. The policy selects one patch (one cell) as the next fixation location. We calculate the 2-D changes of the new fixation location and the current fixation location to have 7 actions (up, down, left, right, …). Please check out the appendix of the paper for more detail.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[soniabaee]

In each frame ,we can have multiple fixations and the policy select one of the patches in each frame. The 2-D changes of the current fixation patch and the selected patch considered as 7 actions (up, down, left, right, …). Note that, we iterate over the fixations of each frame and frames of each video.

[Taoismer]

One more question: I can not totally understand how state representation is form. I know what happen at each frame, but how to transfer historical information between frames. In your paper, it seems that each fixation point receives historical information of corresponding point at the last step. but the paper also points out the number of fixations on each frame is different.

--------------原始邮件-------------- 发件人："soniabaee @.>; 发送时间：2021年3月30日(星期二) 凌晨0:51 收件人："soniabaee/MEDIRL-EyeCar" @.>; 抄送："taoismer @.>;"Author @.>; 主题：Re: [soniabaee/MEDIRL-EyeCar] Where can i get the eyecar video data (#1)

In each frame ,we can have multiple fixations and the policy select one of the patches in each frame. The 2-D changes of the current fixation patch and the selected patch considered as 7 actions (up, down, left, right, …). Note that, we iterate over the fixations of each frame and frames of each video.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[soniabaee]

We are working on a video and each frame of given video can have multiple fixations (all participant watched each video). Therefore to be able to represent historical information, we consider collecting the information of fixation based on each frame and then pass all the information of the previous frame to the next frame in an initialization step. Please let me know if you need pseudocode.

[Taoismer]

It will be the best things if you can provide your pseudocode. Thank you so much!

---

We are working on a video and each frame of given video can have multiple fixations (all participant watched each video). Therefore to be able to represent historical information, we consider collecting the information of fixation based on each frame and then pass all the information of the previous frame to the next frame in an initialization step. Please let me know if you need pseudocode.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

[xinxinlv]

Where can i get the eyecar video data?

hi Taoismer, did you get the raw videos from SHRP2?