search

heal-research / HeuristicLab

HeuristicLab - An environment for heuristic and evolutionary optimization
https://dev.heuristiclab.com
GNU General Public License v3.0
32 stars 15 forks source link

Implement views for statistical hypothesis testing #2031

Closed HeuristicLab-Trac-Bot closed 9 years ago

HeuristicLab-Trac-Bot commented 11 years ago

Issue migrated from trac ticket # 2031

milestone: HeuristicLab 3.3.11 | component: Analysis | priority: medium | resolution: done

2013-04-10 12:28:55: @Shabbafru created the issue

Besides using the charts of the RunCollectionView for analyzing runs there should also be the option to do statistical significance testing. Therefore new RunCollection views should be implemented that offer this functionality.

~~TODO: Check http://powerandsamplesize.com for a better way to do sample size estimation. ~~

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-10 12:29:03: @Shabbafru changed status from new to accepted

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-10 14:07:37: @Shabbafru commented

r9351 added branch for Statistical Hypothesis Testing

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-11 08:55:06: @Shabbafru commented

r9353 initial import of views for statistical testing

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-11 08:55:06: @Shabbafru

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-11 16:52:50: @Shabbafru commented

r9355 added a view that shows boxplots for different sample sizes

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-18 09:47:13: @Shabbafru commented

r9377 added sorting for the chart analysis view

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-18 13:07:29: @Shabbafru commented

r9378 fixed some bugs and the coloring in the chart analysis view

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-18 16:07:02: @Shabbafru commented

r9380 use EnhancedStringConvertibleMatrixView instead of StringConvertibleMatrixView in ResultCorrelationView

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-19 12:02:56: @Shabbafru commented

r9382 the ResultCorrelationView can now also handle parameters and IntValues

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-19 13:02:10: @Shabbafru commented

r9383 ResultCorrelationView is now sortable and some minor cosmetic improvements

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-22 16:57:29: @Shabbafru commented

r9383 reorganized statistical testing ui

HeuristicLab-Trac-Bot commented 11 years ago

2013-04-26 15:46:20: @Shabbafru commented

r9398 fixed a bug in the correlation view

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-15 16:33:32: @Shabbafru commented

r9706

  • added exponential fitting
  • added logarithmic fitting
  • refactored fitting code
  • updated license headers
HeuristicLab-Trac-Bot commented 10 years ago

2013-07-17 11:06:03: @Shabbafru commented

r9712 removed deprecated view

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-17 13:32:16: @Shabbafru commented

r9713

  • made operations in ChartAnalysisView asynchronous
  • renamed views to go along with the other RunCollection views
HeuristicLab-Trac-Bot commented 10 years ago

2013-07-17 16:01:15: @Shabbafru commented

r9717 ported changes from the boxplot view from r9435 to the sample size influence view

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-18 12:42:15: @Shabbafru commented

r9721 added unit tests for the Kruskal Wallis test

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-24 11:24:51: @Shabbafru commented

r9742 added a rich text box dialog for displaying formatted help texts

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-24 19:47:45: @Shabbafru commented

r9749

  • added documentation for chart analysis view
  • some ui improvements
HeuristicLab-Trac-Bot commented 10 years ago

2013-07-26 10:13:10: @Shabbafru commented

r9759 some minor changes

HeuristicLab-Trac-Bot commented 10 years ago

2013-07-30 10:55:34: @Shabbafru commented

r9813 fixed a bug when calculating the percentage of equal groups

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-23 10:33:21: @Shabbafru commented

r9904 updated chart analysis view to work with new progress handling

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-27 10:56:20: @Shabbafru commented

r9908 improved chart analysis view

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-27 11:02:32: @Shabbafru commented

r9909 removed useless split container from chart analysis view

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-27 12:19:51: @Shabbafru commented

r9911

  • renamed statistical run collection views
  • implemented RunCollection events in statistical testing view
  • incorporate content name into view caption
HeuristicLab-Trac-Bot commented 10 years ago

2013-08-27 13:01:44: @Shabbafru commented

r9912 some minor UI improvements

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-27 16:27:35: @Shabbafru commented

r9913

  • redesigned statistical testing view
  • improved sample size influence view
HeuristicLab-Trac-Bot commented 10 years ago

2013-08-29 09:47:20: @Shabbafru commented

r9914 implemented rest of the reviewing comments for the statistical testing view

HeuristicLab-Trac-Bot commented 10 years ago

2013-08-29 12:09:35: @Shabbafru commented

r9917 adapted views to new help system

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-02 13:45:03: @Shabbafru commented

r9922 adapted views to changes in help system

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-02 14:03:51: @Shabbafru commented

r9923 fixed cross threading errors

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-02 15:08:11: @Shabbafru commented

r9925 fixed size of controls in chart analysis view

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-04 17:14:21: @Shabbafru commented

r9936 try to filter NaN values in correlation view and some other minor improvements

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-05 13:28:43: @Shabbafru commented

r9937

  • added an histogram to the StatisticalTestingView
  • don't allow group sizes smaller than 6
HeuristicLab-Trac-Bot commented 10 years ago

2013-09-12 14:21:20: @Shabbafru commented

r9950 added Bonferroni-Holm adjusted p-values to the statistical testing view

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-12 14:53:23: @Shabbafru commented

r9951 updated unit tests

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-13 12:24:11: @Shabbafru commented

r9957 fixed pairwise tests

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-16 14:19:30: @Shabbafru commented

r9968 fixed a bug in the SampleSizeInfluenceView

HeuristicLab-Trac-Bot commented 10 years ago

2013-09-20 20:12:50: @Shabbafru commented

r9998 added confidence intervals to sample size influence view

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-01 21:42:24: @Shabbafru commented

r10016 added recommended sample size to sample size influence view

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 13:49:04: @Shabbafru commented

r10017 improved sample size estimation and some minor improvements

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 13:59:42: @Shabbafru commented

r10018 improved UI of statistical testing view

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 14:24:54: @Shabbafru changed status from accepted to reviewing

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 14:24:54: @Shabbafru changed owner from @Shabbafru to @abeham

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 14:24:54: @Shabbafru commented

Some things to note:

  • I have implemented most of your reviewing comments where it was possible (e.g. the charting changes were not possible).
  • At the moment this is one big plugin. On trunk integration, the views should go to Optimization.Views, the icon to Common.Resources and the additional EnumerableStatisticsExtensions to Common. If the rest of the code is an own plugin or should also go to Common is something we should discuss.
  • I have implemented the estimation of sample sizes as described in several sources. There are 2 types (one "normal" and one for large sample sizes). In my opinion the first one always underestimates the necessary sample sizes, the other one always overestimates sample sizes. I think the problem (though this seems to be no problem in textbooks) is that we do not have an estimator for the variance and confidence intervals. I will have a look into this but I still want to give you the ticket now as everything else is ready for review.
HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 14:24:54: @Shabbafru

HeuristicLab-Trac-Bot commented 10 years ago

2013-10-02 14:43:47: @Shabbafru commented

r10019 added missing static modifiers and removed unused variable

HeuristicLab-Trac-Bot commented 10 years ago

2014-02-11 14:14:22: @Shabbafru changed milestone from HeuristicLab 3.3.x Backlog to HeuristicLab 3.3.10

HeuristicLab-Trac-Bot commented 10 years ago

2014-07-02 13:57:22: @Shabbafru changed milestone from HeuristicLab 3.3.10 to HeuristicLab 3.3.11

HeuristicLab-Trac-Bot commented 10 years ago

2014-07-02 13:57:22: @Shabbafru commented

I'm moving this to 3.3.11 as it won't be ready for the next release.