Fixation Issues

[jgeller112]

Have you done any comparison testing with fixation reports from other trackers? I was thinking about including the saccades package functionality into my gazeR package, but I ran into some worrying results when analyzing fixation data for a VWP. The correspondence between the two was pretty low and the results didn’t look reasonable.

[jgeller112]

As a comparison, here is what data look like using the fixation report from SR:

Here is what it looks like after using saccades:

[tmalsburg]

Hi, of course I have done comparisons with commercial software and also many other tests to see whether the software is working as it's supposed to. I've also used this software in many research projects over the years and never had any problems. Usually the results with this package are similar or better than with other algorithms. In my experience, the differences between algos mostly concern the precision of fixation on- and offsets, not so much location. The differences in your plots, imply differences in location, though (more looks to target with on algo vs the other). Is it possible that the source of these differences is not the saccade detection algorithm but some other step in the processing pipeline? If you believe it's the algorithm, please use the function diagnostic.plot to compare the results of both algorithms. If there is a problem with the fixations produced by this package, it would be great if you could supply some sample data so I can check what's going on.

[jgeller112]

Thanks for the quick response!

You can download sample data here:

install_github("dmirman/gazer")

gaze <- system.file("extdata", "s2.xls", package = "gazer") gaze_samp <- read.table(gaze)

Here is the fixation data supplied by SR:

gaze <- system.file("extdata", "f2.xls", package = "gazer") gaze_fix <-read.table(gaze_fix)

The sccades packages seems to detect many more fixations than SR.

[jgeller112]

This is for one trial using the saccades package

This is for the SR data:

[tmalsburg]

Well, the second plot apparently didn't work, so I can't compare the results. However, the first plot (results from saccades package) looks really nice if you ask me. At least in this limited sample, I can't see any indication that something went wrong. This strengthens my believe that the differences between the two original plots are due to some other problem in the analysis pipeline.

One question: Why do you use fixations in the first place? Isn't it more common in visual world analyses to use the raw samples or time-binned samples (e.g., Barr, 2008)? I don't see what fixations are buying you. In fact they seem to only complicate the analysis. Am I missing something?

[jgeller112]

You are right. In the gazeR package paper we will be recommending the use of sample reports instead of fixations. Some reviewers asked us to include some type of event detection algo that is not proprietary (e.g., SR), which is what brought me here.

On a positive note, I used the saccades package for blink detection in a pupil ds we are using in the paper, and that seemed to work pretty well.

I will keep messing around with it.

[tmalsburg]

Good to hear that blink detection is working. That's a relatively recent addition to the package.

Since, the diagnostic plots don't show any evidence for a problem with saccade/fixation detection, I will close this issue for now. But please don't hesitate to reopen if you find something.