Closed mih closed 5 years ago
Merging #3 into master will not change coverage. The diff coverage is
n/a
.
@@ Coverage Diff @@
## master #3 +/- ##
=======================================
Coverage 88.92% 88.92%
=======================================
Files 8 8
Lines 677 677
=======================================
Hits 602 602
Misses 75 75
Impacted Files | Coverage Δ | |
---|---|---|
remodnav/tests/test_labeled.py | 100% <ø> (ø) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 5690254...f11028f. Read the comment docs.
The difference (in brackets) is less than 3 samples so it should not make much of a difference. For the stats below I truncate the longer one to the size of the shorter one.
looks/sounds good!
One possible conclusion for the observed differences (that are limited to a small subset of stats, while the code to compute them is identical) are differences in the released data vs the originally used data. Another explanation could be a simple copy editing error in the table that was not caught.
sounds like two likely explanations
This is the successor of https://github.com/psychoinformatics-de/remodnav/pull/2 after the re-release of the Anderson et al data.
Some file pairs in the new release have inconsistent length
The difference (in brackets) is less than 3 samples so it should not make much of a difference. For the stats below I truncate the longer one to the size of the shorter one.
Duration stats
No change in the discrepancy pattern from the previous data. Substantial deviations are in BOLD. Values reported in Anderson et al are in parenthesis. Reported values in this section are not based on a re-annotation, but on the annotations released by Anderson et al. There should be no differences.
One possible conclusion for the observed differences (that are limited to a small subset of stats, while the code to compute them is identical) are differences in the released data vs the originally used data. Another explanation could be a simple copy editing error in the table that was not caught.
Fixation durations
Saccade durations
PSO durations
Pursuit durations
Mis-classification summary stats
For all pairwise comparisons, this shows the overall misclassification rate (using timepoints as unit of measure, and limited to timepoints that have been labeled with FIX, SAC, PSO, or PUR by any method, hence ignoring NaN/blinks and undefined (which is rarely used)), same misclassification rate as before, but ignoring PUR events too. The remaining numbers are percentages of labels used in misclassified samples. In contrast to the paper the method label that is misclassifying is given (not "over" and "under", as I found this confusing).
Comparison of human coders comes out very similar to stats reported in Anderson et al. I conclude that the implementation of the evaluation is matching what has been done by them.
images
Analog to table 8 in the paper
img
dots
Analog to table 9 in the paper
videos
Analog to table 10 in the paper