Eye tracking visualization: Eye tracking within the HMD

[bastianilso]

The goal of this visualization is to allow users to gain an understanding of how the eyes have moved around within the headset and some information about gaze confidence.

To do so, we can use the variable LocalGazeDirection X and Y which is a variable going from -1 to 1.

Tasks to do:

• Import data into R.
• Draw a line which visualize each gaze point.
• Identify boundaries of the eye tracking data and draw a polygon around it.
• Use the HMD silhouette as a backdrop in the visualization, to get as close to the mockup as possible.

log2021-07-01 12-50-01.9658Sample.zip

[bastianilso]

@Xelkyo Feel free to use this as the backdrop (the actual visualization space would be within the headset itself.)

Later we can make adjustments to it, if it's too small for the data for example.

[Xelkyo]

I displayed the data from the base of Aldryck's code for the hand movement and adapted it to my use. Here is a screen of what it looks like. Each point is connected to the next one.

I still have to do the outline and I try using something similar to this D3 example : https://observablehq.com/@d3/density-contours/2

[bastianilso]

We need to investigate:

• What are the upper and lower boundaries of localGazeDirection variable. Is it -0.5 to 0.5 or is it -1.0 to 1.0 ?

for creating the polygon, usechull:

D_test = D %>% ungroup() %>% select(x,z) %>% drop_na()

hull_data <- chull(D_test)

D_test_filtered = D_test[hull_data,]

[bastianilso]

D_test_filtered = D_test_filtered %>% 
  mutate(is_hull = TRUE,
         xz = paste(x,z)) %>%
  select(xz, is_hull)

D = D %>% ungroup() %>% mutate(
  xz = paste(x,z)
)

D = D %>% left_join(D_test_filtered, by="xz")

D %>% filter(is_hull) %>% view()

See more about tidyverse: https://www.tidyverse.org/

[Xelkyo]

I tried using chull see the R code below :

data <- subset(data,LocalGazeDirectionX != 'NULL' )

Eyes_Data = data %>% select(LocalGazeDirectionX,LocalGazeDirectionY)
plot(Eyes_Data, cex = 0.5)
Xtrem_points <- chull(Eyes_Data)
Xtrem_points <- c(Xtrem_points, Xtrem_points[1])

D = Eyes_Data[Xtrem_points, ]
D = D %>% mutate(is_hull = TRUE,
  xy = paste(LocalGazeDirectionX,LocalGazeDirectionY))  %>% select(xy,is_hull)

data = data %>% mutate(
  xy = paste(LocalGazeDirectionX,LocalGazeDirectionY)
)

data = data %>% left_join(D, by = "xy")

And rendered it with javascript but the points aren't well placed here it what it renders: (Red lines are the lines that follows the points and green is the inner area)

[bastianilso]

@Xelkyo how does it look if you plot it from R instead, similar to the example provided in ?chull ?

[bastianilso]

https://www.r-bloggers.com/2016/01/convex-hulls-with-codedplyrcode-and-codeggplot2code/

[bastianilso]

Future Works:

• Plot GazeNormal0 and GazeNormal1 as well, center their data in the respective positions for each eye.
• Implement scrubbing, using a similar implementation to aldrycks, and making use of this idea of onion layering.

[Xelkyo]

I managed to fix the chull order with javascript function.

First it calculate the center of all points:

function findCenter(points) {
  var x = 0, y = 0, i, len = points.length;
  for (i = 1; i < len; i++) {
    x += points[i][0];
    y += points[i][1];
  }
  x = x/(len-1)
  y = y/(len-1)
  returnvar = [x,y]

  return (returnvar);   // return average position
}

Then it calculate the angle for each point relative the center:

function findAngles(c, points) {
  var i, len = points.length, p, dx, dy;
  for (i = 1; i < len; i++) {
    p = points[i];
    dx = p[0] - c[0];
    dy = p[1] - c[1];
    p[2] = Math.atan2(dy, dx);
  }
}

I then remove the points that are duplicated through my treatment:

  function removeDuplicate (points){
    var len = points.length;
    var test = [];

    for( var i=2 ; i<len ; i++){
        if ( points[i-1][0] != points[i][0] ) {
            test.push(points[i-1])
        }
      }
      test.push(points[len-1])
      return test
    }

And finally I sort them by their angles:

  function Sort(points){
    var i, j, len = points.length
    for( i=1 ; i<len-1 ; i++){
      for( j=i+1 ; j<len ; j++){
        if ( points[i][2] > points[j][2] ) {
          c = points[i];
          points[i] = points[j];
          points[j] = c;
        }
      }
    }
    return points
  }

And here is the result in image !

[bastianilso]

Cool, please test with multiple participants' data.

Data is in the repository here: https://github.com/bastianilso/WhackAMoleDataJune21

Place data_whack.rda in the base folder of your project.

Loading the data: load('data_whack.rda')

Filtering the data by a specific participant is done like this: D_filtered = D %>% filter(Participant == 7)

Checkout whack_analysis.R for more examples: https://github.com/bastianilso/WhackAMoleDataJune21/blob/main/whackamole_analysis.R#L12

[Xelkyo]

Found interesting things today: https://codepen.io/nihil_9/pen/LYxxRoE https://jsfiddle.net/bfbun6cc/4/ https://d3-graph-gallery.com/graph/density_slider.html

I just realized that the real good keyword for what we need isn't scrubber, scrolling bar or playbar, but it's a slider and so I found some good examples.

[Xelkyo]

@Xelkyo Feel free to use this as the backdrop (the actual visualization space would be within the headset itself.)

Later we can make adjustments to it, if it's too small for the data for example.