stephlj / Traces

Analysis software for single-molecule fluorescence resonance energy transfer (smFRET) data.
MIT License
9 stars 1 forks source link

Residuals: Fix language in manual at least #16

Closed stephlj closed 7 years ago

stephlj commented 8 years ago

Need to be more careful about how I talk about residuals. From Matt: When comparing fits (polynomial vs affine) compare residuals on held out data otherwise polynomial (more params) always does better

mattjj commented 8 years ago

Can you point me to the relevant line in the manual?

This is all a bit pedantic, but it's legit to talk about "residuals" on the data to which a regression was fit. However, those aren't a good metric for goodness-of-fit when comparing regression models of different complexity, like affine vs higher-order polynomial, because as you point out more parameters always do better.

If you have enough data, a simple and very robust metric is to look at residuals on held-out data.

mattjj commented 8 years ago

If you don't have a specific line in mind, and instead you just want to check all the places where residuals are mentioned, let me know.

stephlj commented 7 years ago

Could you take a look at point (9) on pgs 14-15 of the manual, and the caption to figure 7?

mattjj commented 7 years ago

LGTM! (Though you could delete some "Note that" phrases at the starts of sentences without changing their meaning.)

stephlj commented 7 years ago

hey I like my note that's :) Thanks!