Closed HeuristicLab-Trac-Bot closed 9 years ago
I just took a quick look at this. Generally, I'd say this is quite a good addition. Very nice workflow.
The yellow exclamation mark icon should have a tooltip that explains it. I see "Data is normally distributed: !". It seems that the exclamation marks says this is not the case, but at what significance level?p-values should be rounded to 4 places after the point. Otherwise the text box might be too small to fit the whole number: Often the important part (exponent) is at the end. Instead of 2.93559215344167E-05 which would be chopped off to e.g. "2.9355921534416..." it should say 2.9356E-05. The text boxes should be at least as wide to allow these 10 characters to fit nicely.The histogram didn't look to be so useful, maybe a stacked histogram would be more useful with common bins for allThe p-value adjustment needs more explanation. It seems that you have used [correction] for the post-hoc analysis. In the Mann-Whitney-U test most got adjusted from 0.0001 to 0.0007 as you would expect comparing to 7 other groups, but one adjusted p-value remained the same (0.00423...) and another one was adjusted from 0.00025.. to 0.0007. Above all, it seems strange to obtain such p-Values, it rather seems as if this was the significance level.I don't see a possibility to set the significance level.
r11376 fixed RunCollection event handling in ChartAnalysisView
mkommends review comments:
Statistical Testing View:
Check if icons are correct. The label should also change according to the icon and should be also displayed for the tests and not only for normal distribution testAdd links in help view that describe the used algorithms in detailAdjusted p-values are sometimes equal to p-values. Check if this is correctt-Test values should be gray when used with data that is not normally distributedFor the normal distribution check, a check mark should only be given if all are normally distributedInherit StringConvertibleMatrixView and fix the column with or adjust to length of valuesIf an error occurs and the "not enough samples" dialog is shown, the loading bar is not hidden afterwardsIf possible, a kernel density curve should also be displayed in the histogramCheck t-test for one/two sided
Correlations View:
Should display the values as matrix, the method should be chooseableSort inside results and parameters
Chart Analysis View:
Percentiles seem to be switchedChange Relativ Error to Average Relative Errorlog values should be removed from data tableavg/upper/lower values should be descriped in the help text
Sample Size Influence View
add help textused draw without putting back from StatisticsEnumerableOn trunk integration, move algorithms to Analysis plugin, create an own plugin for the views
- fixed column width of p-values
- started working on drawing a normal distribution over the histogram
r11612 fixed a bug in the progress handling of the statistical testing view
r11665 implemented review comments for chart analysis view
r11673 fixed a small bug in Bonferroni-Holm adjustment and added more unit tests for it
- expanded documentation for statistical testing view
- changed t-test to unpooled method
- removed sample size determination for t-test as this is probably not correct
- fixed a bug in Cohens d / Hedges g calculation
- fixed calculation of pairwise tests (no more columns with only zeroes)
- some refactoring
r11693 added dialog for configuring the SignificanceLevel and renamed view
r11695 added more information to the UI about the results of the statistical tests
Replying to [comment:54 ascheibe]:
I did not do graying out the values of the t-test if the data is not normally distributed. It seems that if you have bigger sample sizes it is not such a problem. See http://stats.stackexchange.com/questions/9573/t-test-for-non-normal-when-n50
mkommends review comments: ...
- improved code of statistical testing view
- improved documentation
- fixed confidence intervals calculation
- added more unit tests
- some cosmetic changes
Trunk integrationr11703 moved statistic algs and unit tests to trunk
- moved statistics views to a new plugin (HL.Analysis.Statistics.Views) in trunk
- fixed namespaces of unit tests
r11706 (not migrated) deleted statistical testing branch
Issue migrated from trac ticket # 2031
milestone: HeuristicLab 3.3.11 | component: Analysis | priority: medium | resolution: done
2013-04-10 12:28:55: @Shabbafru created the issue