Revisit summary of engine comparisons

[erex]

Returning to 27Jul23 MCDS.exe testing

$\widehat{N_c}$ differences > 10%

out of 20 data sets each with 13 models fit (260 total) 12 model comparisons (4.6%) resulted in >10% difference

• Cuecount hnherm1 (12% MCDS wins)
• duikercamera hnherm1, hnherm2 (14% mrds wins)
• LTExercise unicos2 (40% MCDS wins), hrpoly2 (35% MCDS wins)
• PTExercise unicos2 (40% MCDS wins), hnherm1 (14% mrds wins)
• savspar80 hrpoly2 (20% MCDS wins)
• savspar81 unicos3 (43% mrds wins) after running on my machine, difference shrinks
• wren5 hnherm1 (15% mrds wins)
• wrencue hrpoly2 (23% MCDS wins)
• datahr1 and datahr2 (I think they're the same) hrpoly2 (56% MCDS wins)

Summary

• half normal with single Hermite polynomial adjustment caused differences in 4 analyses
  • all four analyses are point counts
  • $\Delta \widehat{N_c}$: 12-15% in the four cases
• half normal with 2 Hermite polynomial adjustments caused 14% difference in duiker data set
  • point count again
• uniform with 2 cosine adjustments caused large differences in 2 data sets
  • both data sets are simulated, one lines, one points
• uniform with 3 cosine adjustments caused large differences in a single (savspar81) data set
• hazard rate with 2 simple polynomial adjustments caused large differences in 4 data sets
  • three points count data sets: Savannah sparrow, wren cue, datahr1 (Roccio?)
  • LTExercise has an outlier (all detections<20, except 1 detection at 35) causing problems for adjustments
  • easily dealt with using even the most minimal truncation

Full details here

MCDS-dot-exe-Report-big differences.pdf