arthurPignet commented 3 years ago

MPL accelarations

The fews methods implemented in this section benefit from few improvements, which make them 5th to 10th times faster (even more for FastFedGrads).

The main improvements are:

We build only one model, and never re-build
We benefit from tf.Dataset for the batching, shuffling, and prefetching of data.
We benefit from tf.function for the aggregation step.

[x] Adapt init_model
[x] adapt history
[x] adapt aggregator API
[x] adapt test

codecov-io commented 3 years ago

Codecov Report

Merging #315 (b2962e1) into master (d6b43a2) will decrease coverage by 2.20%. The diff coverage is 65.66%.

@@            Coverage Diff             @@
##           master     #315      +/-   ##
==========================================
- Coverage   82.89%   80.68%   -2.21%     
==========================================
  Files          13       15       +2     
  Lines        2689     3045     +356     
==========================================
+ Hits         2229     2457     +228     
- Misses        460      588     +128

Impacted Files	Coverage Δ
mplc/models.py	`75.94% <40.00%> (-2.43%)`	:arrow_down:
mplc/multi_partner_learning/fast_mpl.py	`61.90% <61.90%> (ø)`
mplc/scenario.py	`82.50% <66.66%> (-0.11%)`	:arrow_down:
mplc/__init__.py	`100.00% <100.00%> (ø)`
mplc/constants.py	`100.00% <100.00%> (ø)`
mplc/contributivity.py	`76.55% <100.00%> (+0.03%)`	:arrow_up:
mplc/multi_partner_learning/__init__.py	`100.00% <100.00%> (ø)`
mplc/multi_partner_learning/basic_mpl.py	`85.26% <100.00%> (ø)`
mplc/multi_partner_learning/utils.py	`86.13% <100.00%> (ø)`
mplc/utils.py	`80.68% <100.00%> (ø)`
... and 4 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update d6b43a2...b2962e1. Read the comment docs.

bowni commented 3 years ago

To keep track of discussions on Slack:

I re-ran the tests I imagined last time on CIFAR10 to compare FedAvg / FastFedAvg / FastFedGrad (full dataset but only 3 epochs and 5 minibatches). FedAvg and FastFedGrad seem to produce logical results, with FastFedGrad being quicker and performing better :pouce: On FastFedAvg though, it doesn't seem to work. It stays at 0.1 and loss is nan.

Logs:

C:\Users\Eric\PycharmProjects\mpl-contributivity\venv\Scripts\python.exe C:/Users/Eric/AppData/Roaming/JetBrains/PyCharmCE2020.3/scratches/scratch_1.py
2021-04-01 10:12:49.551845: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2021-04-01 10:12:49.553196: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2021-04-01 10:12:57.858838: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-04-01 10:12:57.861587: W tensorflow/stream_executor/platform/default/dso_loader.cc:60] Could not load dynamic library 'nvcuda.dll'; dlerror: nvcuda.dll not found
2021-04-01 10:12:57.861947: W tensorflow/stream_executor/cuda/cuda_driver.cc:326] failed call to cuInit: UNKNOWN ERROR (303)
2021-04-01 10:12:57.869861: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:169] retrieving CUDA diagnostic information for host: Laptop-EB
2021-04-01 10:12:57.870317: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:176] hostname: Laptop-EB
2021-04-01 10:12:57.870 | INFO     | mplc.utils:init_gpu_config:107 - No GPU found
2021-04-01 10:13:01 | INFO    | Splitting data among partners: starting now.
2021-04-01 10:13:01 | INFO    | Coherence of config parameters: OK.
2021-04-01 10:13:01 | INFO    | Train data split: starting now.
2021-04-01 10:13:01 | INFO    | Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:01 | INFO    | Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:01 | INFO    | Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:01 | INFO    | Description of data scenario configured:
2021-04-01 10:13:01 | INFO    |    Number of partners defined: 3
2021-04-01 10:13:01 | INFO    |    Data distribution scenario chosen: Random Splitter
2021-04-01 10:13:01 | INFO    |    Multi-partner learning approach: fedavg
2021-04-01 10:13:01 | INFO    |    Weighting option: uniform
2021-04-01 10:13:01 | INFO    |    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:13:01 | INFO    | Data loaded: cifar10
2021-04-01 10:13:01 | INFO    |    45000 train data with 45000 labels
2021-04-01 10:13:01 | INFO    |    5000 val data with 5000 labels
2021-04-01 10:13:01 | INFO    |    10000 test data with 10000 labels
2021-04-01 10:13:06 | INFO    | Splitting data among partners: starting now.
2021-04-01 10:13:06 | INFO    | Coherence of config parameters: OK.
2021-04-01 10:13:06 | INFO    | Train data split: starting now.
2021-04-01 10:13:06 | INFO    | Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:06 | INFO    | Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:06 | INFO    | Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:06 | INFO    | Description of data scenario configured:
2021-04-01 10:13:06 | INFO    |    Number of partners defined: 3
2021-04-01 10:13:06 | INFO    |    Data distribution scenario chosen: Random Splitter
2021-04-01 10:13:06 | INFO    |    Multi-partner learning approach: fast-fedavg
2021-04-01 10:13:06 | INFO    |    Weighting option: uniform
2021-04-01 10:13:06 | INFO    |    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:13:06 | INFO    | Data loaded: cifar10
2021-04-01 10:13:06 | INFO    |    45000 train data with 45000 labels
2021-04-01 10:13:06 | INFO    |    5000 val data with 5000 labels
2021-04-01 10:13:06 | INFO    |    10000 test data with 10000 labels
2021-04-01 10:13:13 | INFO    | Splitting data among partners: starting now.
2021-04-01 10:13:13 | INFO    | Coherence of config parameters: OK.
2021-04-01 10:13:13 | INFO    | Train data split: starting now.
2021-04-01 10:13:13 | INFO    | Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:13 | INFO    | Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:13 | INFO    | Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:13 | INFO    | Description of data scenario configured:
2021-04-01 10:13:13 | INFO    |    Number of partners defined: 3
2021-04-01 10:13:13 | INFO    |    Data distribution scenario chosen: Random Splitter
2021-04-01 10:13:13 | INFO    |    Multi-partner learning approach: fast-fedgrads
2021-04-01 10:13:13 | INFO    |    Weighting option: uniform
2021-04-01 10:13:13 | INFO    |    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:13:13 | INFO    | Data loaded: cifar10
2021-04-01 10:13:13 | INFO    |    45000 train data with 45000 labels
2021-04-01 10:13:13 | INFO    |    5000 val data with 5000 labels
2021-04-01 10:13:13 | INFO    |    10000 test data with 10000 labels
2021-04-01 10:13:13 | INFO    | Now running experiment exp_test_fastmpl_2021-04-01_10h13
2021-04-01 10:13:14.172 | INFO     | mplc.experiment:run:151 - (Experiment exp_test_fastmpl_2021-04-01_10h13) Now starting repeat 1/1
2021-04-01 10:13:14.174 | INFO     | mplc.experiment:run:156 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1) Now running scenario 1/3
2021-04-01 10:13:14.176 | INFO     | mplc.splitter:split:37 - Splitting data among partners: starting now.
2021-04-01 10:13:14.179 | INFO     | mplc.splitter:split:39 - Coherence of config parameters: OK.
2021-04-01 10:13:14.180 | INFO     | mplc.splitter:split:41 - Train data split: starting now.
2021-04-01 10:13:21.574 | INFO     | mplc.splitter:split:54 - Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:21.634 | INFO     | mplc.splitter:split:54 - Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:21.682 | INFO     | mplc.splitter:split:54 - Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:13:21.695 | INFO     | mplc.scenario:log_scenario_description:416 - Description of data scenario configured:
2021-04-01 10:13:21.695 | INFO     | mplc.scenario:log_scenario_description:417 -    Number of partners defined: 3
2021-04-01 10:13:21.696 | INFO     | mplc.scenario:log_scenario_description:418 -    Data distribution scenario chosen: Random Splitter
2021-04-01 10:13:21.696 | INFO     | mplc.scenario:log_scenario_description:419 -    Multi-partner learning approach: fedavg
2021-04-01 10:13:21.697 | INFO     | mplc.scenario:log_scenario_description:420 -    Weighting option: uniform
2021-04-01 10:13:21.698 | INFO     | mplc.scenario:log_scenario_description:421 -    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:13:21.698 | INFO     | mplc.scenario:log_scenario_description:427 - Data loaded: cifar10
2021-04-01 10:13:21.698 | INFO     | mplc.scenario:log_scenario_description:432 -    45000 train data with 45000 labels
2021-04-01 10:13:21.699 | INFO     | mplc.scenario:log_scenario_description:435 -    5000 val data with 5000 labels
2021-04-01 10:13:21.699 | INFO     | mplc.scenario:log_scenario_description:438 -    10000 test data with 10000 labels
2021-04-01 10:13:43.064 | INFO     | mplc.scenario:run:567 - Now starting running scenario scenario_0_repeat_0_2021-04-01_10h13_eeb
2021-04-01 10:13:43.260094: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-04-01 10:13:43.269055: I tensorflow/compiler/jit/xla_gpu_device.cc:99] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-04-01 10:13:43.650 | INFO     | mplc.multi_partner_learning.basic_mpl:init_model:129 - Init new model
2021-04-01 10:13:43.655 | INFO     | mplc.multi_partner_learning.basic_mpl:__init__:84 - ## Preparation of model's training on partners with ids: ['#0', '#1', '#2']
2021-04-01 10:13:43.906 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°0 of epoch n°0, init each partner's models with a copy of the global model
2021-04-01 10:13:45.034194: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:116] None of the MLIR optimization passes are enabled (registered 2)
2021-04-01 10:14:24.110 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°1 of epoch n°0, init each partner's models with a copy of the global model
2021-04-01 10:15:01.485 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°2 of epoch n°0, init each partner's models with a copy of the global model
2021-04-01 10:15:39.099 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°3 of epoch n°0, init each partner's models with a copy of the global model
2021-04-01 10:16:16.002 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°4 of epoch n°0, init each partner's models with a copy of the global model
2021-04-01 10:16:19.771 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_model_val_perf:191 -    Model evaluation at the end of the epoch 00/02: ['1.957', '0.275']
2021-04-01 10:16:53.790 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°0 of epoch n°1, init each partner's models with a copy of the global model
2021-04-01 10:17:28.551 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°1 of epoch n°1, init each partner's models with a copy of the global model
2021-04-01 10:18:02.407 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°2 of epoch n°1, init each partner's models with a copy of the global model
2021-04-01 10:18:36.802 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°3 of epoch n°1, init each partner's models with a copy of the global model
2021-04-01 10:19:13.634 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°4 of epoch n°1, init each partner's models with a copy of the global model
2021-04-01 10:19:17.034 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_model_val_perf:191 -    Model evaluation at the end of the epoch 01/02: ['1.728', '0.385']
2021-04-01 10:19:52.606 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°0 of epoch n°2, init each partner's models with a copy of the global model
2021-04-01 10:20:31.887 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°1 of epoch n°2, init each partner's models with a copy of the global model
2021-04-01 10:21:12.724 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°2 of epoch n°2, init each partner's models with a copy of the global model
2021-04-01 10:21:55.766 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°3 of epoch n°2, init each partner's models with a copy of the global model
2021-04-01 10:22:38.775 | INFO     | mplc.multi_partner_learning.basic_mpl:fit_minibatch:374 - (fedavg) Minibatch n°4 of epoch n°2, init each partner's models with a copy of the global model
2021-04-01 10:22:43.117 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_model_val_perf:191 -    Model evaluation at the end of the epoch 02/02: ['1.635', '0.433']
2021-04-01 10:23:26.641 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_final_model_test_perf:196 - ### Evaluating model on test data:
2021-04-01 10:23:35.788 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_final_model_test_perf:219 -    Model metrics names: ['loss', 'accuracy']
2021-04-01 10:23:35.789 | INFO     | mplc.multi_partner_learning.basic_mpl:eval_and_log_final_model_test_perf:220 -    Model metrics values: ['1.605', '0.440']
2021-04-01 10:23:35.790 | INFO     | mplc.multi_partner_learning.basic_mpl:fit:264 - Training and evaluation on multiple partners: done. (592.132 seconds)
C:\Users\Eric\PycharmProjects\mpl-contributivity\venv\lib\site-packages\numpy\core\_asarray.py:136: VisibleDeprecationWarning: Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray
  return array(a, dtype, copy=False, order=order, subok=True)
2021-04-01 10:23:37.424 | INFO     | mplc.experiment:run:175 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1, scenario 1/3) Results saved to results.csv in folder experiments\exp_test_fastmpl_2021-04-01_10h13
2021-04-01 10:23:37.430 | INFO     | mplc.experiment:run:156 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1) Now running scenario 2/3
2021-04-01 10:23:37.432 | INFO     | mplc.splitter:split:37 - Splitting data among partners: starting now.
2021-04-01 10:23:37.433 | INFO     | mplc.splitter:split:39 - Coherence of config parameters: OK.
2021-04-01 10:23:37.434 | INFO     | mplc.splitter:split:41 - Train data split: starting now.
2021-04-01 10:23:44.264 | INFO     | mplc.splitter:split:54 - Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:23:44.266 | INFO     | mplc.splitter:split:54 - Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:23:44.267 | INFO     | mplc.splitter:split:54 - Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:23:44.269 | INFO     | mplc.scenario:log_scenario_description:416 - Description of data scenario configured:
2021-04-01 10:23:44.269 | INFO     | mplc.scenario:log_scenario_description:417 -    Number of partners defined: 3
2021-04-01 10:23:44.270 | INFO     | mplc.scenario:log_scenario_description:418 -    Data distribution scenario chosen: Random Splitter
2021-04-01 10:23:44.270 | INFO     | mplc.scenario:log_scenario_description:419 -    Multi-partner learning approach: fast-fedavg
2021-04-01 10:23:44.271 | INFO     | mplc.scenario:log_scenario_description:420 -    Weighting option: uniform
2021-04-01 10:23:44.271 | INFO     | mplc.scenario:log_scenario_description:421 -    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:23:44.272 | INFO     | mplc.scenario:log_scenario_description:427 - Data loaded: cifar10
2021-04-01 10:23:44.272 | INFO     | mplc.scenario:log_scenario_description:432 -    45000 train data with 45000 labels
2021-04-01 10:23:44.272 | INFO     | mplc.scenario:log_scenario_description:435 -    5000 val data with 5000 labels
2021-04-01 10:23:44.273 | INFO     | mplc.scenario:log_scenario_description:438 -    10000 test data with 10000 labels
2021-04-01 10:23:59.161 | INFO     | mplc.scenario:run:567 - Now starting running scenario scenario_1_repeat_0_2021-04-01_10h23_b2f
2021-04-01 10:23:59.161 | INFO     | mplc.multi_partner_learning.fast_mpl:__init__:74 - ## Preparation of model's training on partners with ids: ['#0', '#1', '#2']
2021-04-01 10:24:03.418 | INFO     | mplc.multi_partner_learning.fast_mpl:init_model:141 - Init new model
2021-04-01 10:26:17.723 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedAvg] > Epoch 1 /3 - 134.0 s. > loss: 7.213101678560291e+20 -- accuracy: 0.26  -- val_loss: 2.0239431129062216e+27 -- val_accuracy: 0.29 
2021-04-01 10:28:11.707 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedAvg] > Epoch 2 /3 - 114.0 s. > loss: nan   -- accuracy: 0.11  -- val_loss: nan   -- val_accuracy: 0.1  
2021-04-01 10:30:04.673 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedAvg] > Epoch 3 /3 - 113.0 s. > loss: nan   -- accuracy: 0.1   -- val_loss: nan   -- val_accuracy: 0.1  
2021-04-01 10:30:11.563 | INFO     | mplc.multi_partner_learning.fast_mpl:log_end_training:174 - [FastFedAvg] > Training 3 epoch in 361.242 seconds. Tests scores: loss: nan -- accuracy: 0.1
2021-04-01 10:30:11.889 | INFO     | mplc.experiment:run:175 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1, scenario 2/3) Results saved to results.csv in folder experiments\exp_test_fastmpl_2021-04-01_10h13
2021-04-01 10:30:11.896 | INFO     | mplc.experiment:run:156 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1) Now running scenario 3/3
2021-04-01 10:30:11.899 | INFO     | mplc.splitter:split:37 - Splitting data among partners: starting now.
2021-04-01 10:30:11.900 | INFO     | mplc.splitter:split:39 - Coherence of config parameters: OK.
2021-04-01 10:30:11.900 | INFO     | mplc.splitter:split:41 - Train data split: starting now.
2021-04-01 10:30:18.931 | INFO     | mplc.splitter:split:54 - Partner #0: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:30:18.932 | INFO     | mplc.splitter:split:54 - Partner #1: 18900 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:30:18.934 | INFO     | mplc.splitter:split:54 - Partner #2: 7200 samples with labels [0 1 2 3 4 5 6 7 8 9]
2021-04-01 10:30:18.936 | INFO     | mplc.scenario:log_scenario_description:416 - Description of data scenario configured:
2021-04-01 10:30:18.936 | INFO     | mplc.scenario:log_scenario_description:417 -    Number of partners defined: 3
2021-04-01 10:30:18.937 | INFO     | mplc.scenario:log_scenario_description:418 -    Data distribution scenario chosen: Random Splitter
2021-04-01 10:30:18.937 | INFO     | mplc.scenario:log_scenario_description:419 -    Multi-partner learning approach: fast-fedgrads
2021-04-01 10:30:18.938 | INFO     | mplc.scenario:log_scenario_description:420 -    Weighting option: uniform
2021-04-01 10:30:18.939 | INFO     | mplc.scenario:log_scenario_description:421 -    Iterations parameters: 3 epochs > 5 mini-batches > 8 gradient updates per pass
2021-04-01 10:30:18.939 | INFO     | mplc.scenario:log_scenario_description:427 - Data loaded: cifar10
2021-04-01 10:30:18.939 | INFO     | mplc.scenario:log_scenario_description:432 -    45000 train data with 45000 labels
2021-04-01 10:30:18.940 | INFO     | mplc.scenario:log_scenario_description:435 -    5000 val data with 5000 labels
2021-04-01 10:30:18.941 | INFO     | mplc.scenario:log_scenario_description:438 -    10000 test data with 10000 labels
2021-04-01 10:30:32.075 | INFO     | mplc.scenario:run:567 - Now starting running scenario scenario_2_repeat_0_2021-04-01_10h30_8f8
2021-04-01 10:30:32.076 | INFO     | mplc.multi_partner_learning.fast_mpl:__init__:74 - ## Preparation of model's training on partners with ids: ['#0', '#1', '#2']
2021-04-01 10:30:34.294 | INFO     | mplc.multi_partner_learning.fast_mpl:init_model:141 - Init new model
2021-04-01 10:32:33.278 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedGrad] > Epoch 1 /3 - 118.0 s. > loss: 1.9   -- accuracy: 0.31  -- val_loss: 1.64  -- val_accuracy: 0.41 
2021-04-01 10:34:39.209 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedGrad] > Epoch 2 /3 - 126.0 s. > loss: 1.54  -- accuracy: 0.45  -- val_loss: 1.44  -- val_accuracy: 0.48 
2021-04-01 10:36:52.921 | INFO     | mplc.multi_partner_learning.fast_mpl:log_epoch:161 - [FastFedGrad] > Epoch 3 /3 - 134.0 s. > loss: 1.38  -- accuracy: 0.51  -- val_loss: 1.31  -- val_accuracy: 0.54 
2021-04-01 10:36:59.881 | INFO     | mplc.multi_partner_learning.fast_mpl:log_end_training:174 - [FastFedGrad] > Training 3 epoch in 378.105 seconds. Tests scores: loss: 1.312 -- accuracy: 0.535
2021-04-01 10:36:59.920 | INFO     | mplc.experiment:run:175 - (Experiment exp_test_fastmpl_2021-04-01_10h13, repeat 1/1, scenario 3/3) Results saved to results.csv in folder experiments\exp_test_fastmpl_2021-04-01_10h13

LabeliaLabs / distributed-learning-contributivity

Faster versions of FedAvg, FedGrads and Smodel. #315

MPL accelarations

Codecov Report