Closed kegl closed 3 years ago
Merging #245 into advanced will increase coverage by
0.43%
. The diff coverage is98.50%
.
@@ Coverage Diff @@
## advanced #245 +/- ##
============================================
+ Coverage 81.75% 82.18% +0.43%
============================================
Files 133 138 +5
Lines 4921 5040 +119
============================================
+ Hits 4023 4142 +119
Misses 898 898
Impacted Files | Coverage Δ | |
---|---|---|
rampwf/utils/cli/testing.py | 0.00% <0.00%> (ø) |
|
rampwf/tests/kits/iris_data_label/problem.py | 100.00% <100.00%> (ø) |
|
...label/submissions/random_forest_10_10/estimator.py | 100.00% <100.00%> (ø) |
|
...s_data_label/submissions/starting_kit/estimator.py | 100.00% <100.00%> (ø) |
|
rampwf/tests/test_kits.py | 96.15% <100.00%> (+0.59%) |
:arrow_up: |
rampwf/utils/testing.py | 100.00% <100.00%> (ø) |
|
rampwf/score_types/base.py | 69.23% <0.00%> (-4.11%) |
:arrow_down: |
rampwf/score_types/roc_auc.py | 100.00% <0.00%> (ø) |
|
rampwf/score_types/combined.py | 91.66% <0.00%> (ø) |
|
rampwf/score_types/brier_score.py | 100.00% <0.00%> (ø) |
|
... and 16 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update b92ca38...8958b7a. Read the comment docs.
how about you just have a data_path param that defaults to "./data" so we know we don't break anything?
We are putting the different data in ./data/<data_label>/
, (better organization than flat ./<data_label>/
), otherwise it's the same: the data label defaults to ''
which then handled by the script in the usual way, essentially means the data path defaults to ./data
.
You will see no difference in the behavior if you use ramp-test
without data label.
We thought this through, trying different setups (we've bee using this internally for 8 months now), and this was the best.
We add an optional
--data-label
switch toramp-test
.If specified
problem.get_train_data
andproblem.get_train_data
so the user can load the appropriate data, and--save-output
, the training output is saved in subfolders submissions/If not specified, the behavior of
ramp-test
will remain the same. This also means that it will not interfere with ramp-board (@agramfort).To close #244