EducationalTestingService / skll

SciKit-Learn Laboratory (SKLL) makes it easy to run machine learning experiments.
http://skll.readthedocs.org
Other
551 stars 67 forks source link

Use 5 folds for grid search #667

Closed desilinguist closed 3 years ago

desilinguist commented 3 years ago

This PR closes #487.

See issue for motivation to change the default number of folds for grid search cross-validation. One by-product of the change is that experiments will certainly take longer to run but they are likely to be more reliable.

Changes:

codecov[bot] commented 3 years ago

Codecov Report

Merging #667 (ea4966d) into main (dc5ed05) will not change coverage. The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##             main     #667   +/-   ##
=======================================
  Coverage   96.76%   96.76%           
=======================================
  Files          63       63           
  Lines        9077     9077           
=======================================
  Hits         8783     8783           
  Misses        294      294           
Impacted Files Coverage Δ
skll/config/__init__.py 95.11% <ø> (ø)
skll/learner/__init__.py 97.09% <ø> (ø)
skll/learner/voting.py 98.42% <ø> (ø)
tests/test_input.py 99.79% <ø> (ø)
tests/test_output.py 100.00% <ø> (ø)
tests/test_voting_learners_api_1.py 100.00% <ø> (ø)
tests/test_voting_learners_api_2.py 100.00% <ø> (ø)
tests/test_voting_learners_api_3.py 100.00% <ø> (ø)
tests/test_voting_learners_api_5.py 100.00% <ø> (ø)
tests/test_voting_learners_expts_1.py 98.38% <ø> (ø)
... and 6 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update dc5ed05...ea4966d. Read the comment docs.

desilinguist commented 3 years ago

@mulhod it'd be really great if you could test this with gridmap as well. Thanks!