LabeliaLabs / distributed-learning-contributivity

Simulate collaborative ML scenarios, experiment multi-partner learning approaches and measure respective contributions of different datasets to model performance.
https://www.labelia.org
Apache License 2.0
57 stars 12 forks source link

Create a fully flexible splitter #320

Closed bowni closed 3 years ago

bowni commented 3 years ago

The fully flexible splitter enables the definition of a very precise split of a dataset among partners. One can indicate for each partner the % of data samples of each class that the partner should receive.

codecov-io commented 3 years ago

Codecov Report

Merging #320 (91f915d) into master (2e28840) will decrease coverage by 0.99%. The diff coverage is 17.07%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #320      +/-   ##
==========================================
- Coverage   82.63%   81.64%   -1.00%     
==========================================
  Files          13       13              
  Lines        2637     2669      +32     
==========================================
  Hits         2179     2179              
- Misses        458      490      +32     
Impacted Files Coverage Δ
mplc/splitter.py 81.53% <13.51%> (-14.76%) :arrow_down:
mplc/scenario.py 82.60% <50.00%> (-0.48%) :arrow_down:
mplc/dataset.py 85.83% <0.00%> (-0.21%) :arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2e28840...c08e5e2. Read the comment docs.

bowni commented 3 years ago

What do you think of rewrite random and stratified on the top of this method ?

Agreed! But maybe not in this PR. It could be an enhancement (or refacto let's say) afterwards.

bowni commented 3 years ago

I will have only one question: can a flexible splitter be used with a config file ? Did you test it ?

Good point - just did it, enabled to catch an issue when the Scenario copies the Splitter 👍

bowni commented 3 years ago

@RomainGoussault could you check if your "requested changes" are now OK?