Closed arthurPignet closed 3 years ago
I don't think we need the full track of which line of which function of which module triggers what log on every-day-utilisation
2021-01-14 14:06:09.129 | INFO | mplc.splitter:split:44 - ### Splitting data among partners: 2021-01-14 14:06:09.130 | INFO | mplc.splitter:split:45 - Train data split: 2021-01-14 14:06:09.273 | INFO | mplc.splitter:split:58 - Partner #0: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.274 | INFO | mplc.splitter:split:58 - Partner #1: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.276 | INFO | mplc.splitter:split:58 - Partner #2: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.278 | INFO | mplc.splitter:split:58 - Partner #3: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.281 | INFO | mplc.splitter:split:58 - Partner #4: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.282 | INFO | mplc.splitter:split:58 - Partner #5: 4500 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.285 | INFO | mplc.splitter:split:58 - Partner #6: 4499 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.287 | INFO | mplc.splitter:split:58 - Partner #7: 4501 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.290 | INFO | mplc.splitter:split:58 - Partner #8: 4499 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.292 | INFO | mplc.splitter:split:58 - Partner #9: 4501 samples with labels [0 1 2 3 4 5 6 7 8 9] 2021-01-14 14:06:09.295 | INFO | mplc.scenario:log_scenario_description:407 - Description of data scenario configured: 2021-01-14 14:06:09.297 | INFO | mplc.scenario:log_scenario_description:408 - Number of partners defined: 10 2021-01-14 14:06:09.298 | INFO | mplc.scenario:log_scenario_description:409 - Data distribution scenario chosen: Random samples split 2021-01-14 14:06:09.301 | INFO | mplc.scenario:log_scenario_description:410 - Multi-partner learning approach: fedavg 2021-01-14 14:06:09.302 | INFO | mplc.scenario:log_scenario_description:411 - Weighting option: data-volume 2021-01-14 14:06:09.302 | INFO | mplc.scenario:log_scenario_description:412 - Iterations parameters: 20 epochs > 1 mini-batches > 32 gradient updates per pass 2021-01-14 14:06:09.303 | INFO | mplc.scenario:log_scenario_description:418 - Data loaded: cifar10 2021-01-14 14:06:09.306 | INFO | mplc.scenario:log_scenario_description:423 - 45000 train data with 45000 labels 2021-01-14 14:06:09.307 | INFO | mplc.scenario:log_scenario_description:426 - 5000 val data with 5000 labels 2021-01-14 14:06:09.307 | INFO | mplc.scenario:log_scenario_description:429 - 10000 test data with 10000 labels 2021-01-14 14:06:09.307 | INFO | __main__:__init__:31 - ## Preparation of model's training on partners with ids: ['#0', '#1', '#2', '#3', '#4', '#5', '#6', '#7', '#8', '#9'] 2021-01-14 14:06:09.816 | INFO | mplc.multi_partner_learning:init_model:130 - Init new model 2021-01-14 14:06:16.587 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 1 /20 - 7.0 s . loss: 2.05 -- accuracy: 0.24 -- val_loss: 2.18 -- val_accuracy: 0.22 2021-01-14 14:06:23.079 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 2 /20 - 6.0 s . loss: 1.89 -- accuracy: 0.31 -- val_loss: 1.96 -- val_accuracy: 0.28 2021-01-14 14:06:29.564 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 3 /20 - 6.0 s . loss: 1.92 -- accuracy: 0.32 -- val_loss: 1.98 -- val_accuracy: 0.34 2021-01-14 14:06:35.944 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 4 /20 - 6.0 s . loss: 1.85 -- accuracy: 0.35 -- val_loss: 2.55 -- val_accuracy: 0.33 2021-01-14 14:06:42.520 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 5 /20 - 7.0 s . loss: 1.85 -- accuracy: 0.36 -- val_loss: 2.74 -- val_accuracy: 0.29 2021-01-14 14:06:48.788 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 6 /20 - 6.0 s . loss: 1.77 -- accuracy: 0.39 -- val_loss: 2.01 -- val_accuracy: 0.34 2021-01-14 14:06:55.094 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 7 /20 - 6.0 s . loss: 1.63 -- accuracy: 0.41 -- val_loss: 1.99 -- val_accuracy: 0.32 2021-01-14 14:07:01.306 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 8 /20 - 6.0 s . loss: 1.61 -- accuracy: 0.42 -- val_loss: 1.92 -- val_accuracy: 0.33 2021-01-14 14:07:07.617 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 9 /20 - 6.0 s . loss: 1.55 -- accuracy: 0.44 -- val_loss: 1.62 -- val_accuracy: 0.4 2021-01-14 14:07:13.849 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 10/20 - 6.0 s . loss: 1.49 -- accuracy: 0.46 -- val_loss: 1.49 -- val_accuracy: 0.45 2021-01-14 14:07:20.303 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 11/20 - 6.0 s . loss: 1.44 -- accuracy: 0.48 -- val_loss: 1.44 -- val_accuracy: 0.48 2021-01-14 14:07:26.502 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 12/20 - 6.0 s . loss: 1.4 -- accuracy: 0.5 -- val_loss: 1.42 -- val_accuracy: 0.49 2021-01-14 14:07:32.827 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 13/20 - 6.0 s . loss: 1.37 -- accuracy: 0.51 -- val_loss: 1.4 -- val_accuracy: 0.49 2021-01-14 14:07:39.063 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 14/20 - 6.0 s . loss: 1.33 -- accuracy: 0.53 -- val_loss: 1.38 -- val_accuracy: 0.5 2021-01-14 14:07:45.367 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 15/20 - 6.0 s . loss: 1.3 -- accuracy: 0.54 -- val_loss: 1.33 -- val_accuracy: 0.52 2021-01-14 14:07:51.717 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 16/20 - 6.0 s . loss: 1.26 -- accuracy: 0.55 -- val_loss: 1.26 -- val_accuracy: 0.56 2021-01-14 14:07:58.175 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 17/20 - 6.0 s . loss: 1.23 -- accuracy: 0.56 -- val_loss: 1.22 -- val_accuracy: 0.57 2021-01-14 14:08:04.518 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 18/20 - 6.0 s . loss: 1.2 -- accuracy: 0.57 -- val_loss: 1.19 -- val_accuracy: 0.58 2021-01-14 14:08:10.916 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 19/20 - 6.0 s . loss: 1.17 -- accuracy: 0.58 -- val_loss: 1.17 -- val_accuracy: 0.59 2021-01-14 14:08:17.175 | INFO | __main__:log_epoch:83 - [FedGDOf] > Epoch 20/20 - 6.0 s . loss: 1.14 -- accuracy: 0.6 -- val_loss: 1.14 -- val_accuracy: 0.6 2021-01-14 14:08:17.561 | INFO | __main__:fit:149 - [FedGDOf] > Training 20 in 127.309 secondes. Tests scores: loss: 1.134 -- accuracy: 0.601
2021-01-14 17:54:30 | INFO | ### Splitting data among partners: 2021-01-14 17:54:30 | INFO | Train data split: 2021-01-14 17:54:36 | INFO | Partner #0: 9000 samples with labels [8 9] 2021-01-14 17:54:36 | INFO | Partner #1: 9000 samples with labels [6 7] 2021-01-14 17:54:36 | INFO | Partner #2: 9000 samples with labels [4 5] 2021-01-14 17:54:36 | INFO | Partner #3: 9000 samples with labels [2 3] 2021-01-14 17:54:36 | INFO | Partner #4: 9000 samples with labels [0 1] 2021-01-14 17:54:36 | INFO | Description of data scenario configured: 2021-01-14 17:54:36 | INFO | Number of partners defined: 5 2021-01-14 17:54:36 | INFO | Data distribution scenario chosen: Stratified samples split 2021-01-14 17:54:36 | INFO | Multi-partner learning approach: fedavg 2021-01-14 17:54:36 | INFO | Weighting option: data-volume 2021-01-14 17:54:36 | INFO | Iterations parameters: 20 epochs > 1 mini-batches > 32 gradient updates per pass 2021-01-14 17:54:36 | INFO | Data loaded: cifar10 2021-01-14 17:54:36 | INFO | 45000 train data with 45000 labels 2021-01-14 17:54:36 | INFO | 5000 val data with 5000 labels 2021-01-14 17:54:36 | INFO | 10000 test data with 10000 labels 2021-01-14 17:54:36 | INFO | ## Preparation of model's training on partners with ids: ['#0', '#1', '#2', '#3', '#4'] 2021-01-14 17:54:37 | INFO | Init new model 2021-01-14 17:54:41 | INFO | [FedGDOf] > Epoch 1 /20 - 5.0 s . loss: 0.78 -- accuracy: 0.57 -- val_loss: 2.31 -- val_accuracy: 0.11 2021-01-14 17:54:46 | INFO | [FedGDOf] > Epoch 2 /20 - 4.0 s . loss: 0.76 -- accuracy: 0.62 -- val_loss: 2.31 -- val_accuracy: 0.11 2021-01-14 17:54:50 | INFO | [FedGDOf] > Epoch 3 /20 - 4.0 s . loss: 0.63 -- accuracy: 0.69 -- val_loss: 2.39 -- val_accuracy: 0.16 2021-01-14 17:54:54 | INFO | [FedGDOf] > Epoch 4 /20 - 4.0 s . loss: 0.61 -- accuracy: 0.71 -- val_loss: 2.73 -- val_accuracy: 0.18 2021-01-14 17:54:58 | INFO | [FedGDOf] > Epoch 5 /20 - 4.0 s . loss: 0.6 -- accuracy: 0.72 -- val_loss: 2.31 -- val_accuracy: 0.22 2021-01-14 17:55:02 | INFO | [FedGDOf] > Epoch 6 /20 - 4.0 s . loss: 0.6 -- accuracy: 0.73 -- val_loss: 2.52 -- val_accuracy: 0.19 2021-01-14 17:55:06 | INFO | [FedGDOf] > Epoch 7 /20 - 4.0 s . loss: 0.58 -- accuracy: 0.74 -- val_loss: 2.21 -- val_accuracy: 0.22 2021-01-14 17:55:10 | INFO | [FedGDOf] > Epoch 8 /20 - 4.0 s . loss: 0.53 -- accuracy: 0.76 -- val_loss: 2.12 -- val_accuracy: 0.22 2021-01-14 17:55:14 | INFO | [FedGDOf] > Epoch 9 /20 - 4.0 s . loss: 0.52 -- accuracy: 0.77 -- val_loss: 2.1 -- val_accuracy: 0.23 2021-01-14 17:55:18 | INFO | [FedGDOf] > Epoch 10/20 - 4.0 s . loss: 0.51 -- accuracy: 0.78 -- val_loss: 2.08 -- val_accuracy: 0.23 2021-01-14 17:55:22 | INFO | [FedGDOf] > Epoch 11/20 - 4.0 s . loss: 0.5 -- accuracy: 0.78 -- val_loss: 2.06 -- val_accuracy: 0.24 2021-01-14 17:55:26 | INFO | [FedGDOf] > Epoch 12/20 - 4.0 s . loss: 0.48 -- accuracy: 0.79 -- val_loss: 2.03 -- val_accuracy: 0.27 2021-01-14 17:55:30 | INFO | [FedGDOf] > Epoch 13/20 - 4.0 s . loss: 0.46 -- accuracy: 0.8 -- val_loss: 1.97 -- val_accuracy: 0.31 2021-01-14 17:55:34 | INFO | [FedGDOf] > Epoch 14/20 - 4.0 s . loss: 0.44 -- accuracy: 0.81 -- val_loss: 1.97 -- val_accuracy: 0.31 2021-01-14 17:55:38 | INFO | [FedGDOf] > Epoch 15/20 - 4.0 s . loss: 0.44 -- accuracy: 0.81 -- val_loss: 2.02 -- val_accuracy: 0.28 2021-01-14 17:55:43 | INFO | [FedGDOf] > Epoch 16/20 - 4.0 s . loss: 0.42 -- accuracy: 0.82 -- val_loss: 2.05 -- val_accuracy: 0.24 2021-01-14 17:55:47 | INFO | [FedGDOf] > Epoch 17/20 - 4.0 s . loss: 0.42 -- accuracy: 0.82 -- val_loss: 2.03 -- val_accuracy: 0.25 2021-01-14 17:55:51 | INFO | [FedGDOf] > Epoch 18/20 - 4.0 s . loss: 0.4 -- accuracy: 0.83 -- val_loss: 1.95 -- val_accuracy: 0.29 2021-01-14 17:55:55 | INFO | [FedGDOf] > Epoch 19/20 - 4.0 s . loss: 0.4 -- accuracy: 0.83 -- val_loss: 1.91 -- val_accuracy: 0.32 2021-01-14 17:55:59 | INFO | [FedGDOf] > Epoch 20/20 - 4.0 s . loss: 0.4 -- accuracy: 0.83 -- val_loss: 1.92 -- val_accuracy: 0.32 2021-01-14 17:55:59 | INFO | [FedGDOf] > Training 20 in 82.235 secondes. Tests scores: loss: 1.912 -- accuracy: 0.318
By the way the scenario logs are quite a mess, sometime with ##, sometime without. I create an issue about that
I don't think we need the full track of which line of which function of which module triggers what log on every-day-utilisation
old log
new log
By the way the scenario logs are quite a mess, sometime with ##, sometime without. I create an issue about that