Closed MicheleCancilla closed 3 years ago
DataGenerator sometimes skips some batches:
Starting validation: Validation: Epoch 0/149 (batch 0/24) - categorical_accuracy: 0.85 Validation: Epoch 0/149 (batch 1/24) - categorical_accuracy: 0.55 Validation: Epoch 0/149 (batch 2/24) - categorical_accuracy: 0.325 Validation: Epoch 0/149 (batch 3/24) - categorical_accuracy: 0.45 Validation: Epoch 0/149 (batch 4/24) - categorical_accuracy: 0.425 Validation: Epoch 0/149 (batch 5/24) - categorical_accuracy: 0.425 Validation: Epoch 0/149 (batch 6/24) - categorical_accuracy: 0.95 Validation: Epoch 0/149 (batch 7/24) - categorical_accuracy: 0.9 Validation: Epoch 0/149 (batch 8/24) - categorical_accuracy: 0.9 Validation: Epoch 0/149 (batch 9/24) - categorical_accuracy: 0.775 Validation: Epoch 0/149 (batch 10/24) - categorical_accuracy: 0.925 Validation: Epoch 0/149 (batch 11/24) - categorical_accuracy: 0.875 Validation: Epoch 0/149 (batch 12/24) - categorical_accuracy: 0.85 Validation: Epoch 0/149 (batch 13/24) - categorical_accuracy: 0.95 Validation: Epoch 0/149 (batch 14/24) - categorical_accuracy: 0.95 Validation: Epoch 0/149 (batch 15/24) - categorical_accuracy: 0.9 Validation: Epoch 0/149 (batch 16/24) - categorical_accuracy: 0.7 Validation: Epoch 0/149 (batch 17/24) - categorical_accuracy: 0.65 Validation: Epoch 0/149 (batch 18/24) - categorical_accuracy: 0.675 Validation: Epoch 0/149 (batch 19/24) - categorical_accuracy: 0.6 Validation: Epoch 0/149 (batch **20/24**) - categorical_accuracy: 0.275 <-- 21-24 have been skipped Validation categorical accuracy: 0.709524
Or it saves them for later:
Starting validation: Validation: Epoch 1/149 (batch 0/24) - categorical_accuracy: 0.15 Validation: Epoch 1/149 (batch 1/24) - categorical_accuracy: 0.25 Validation: Epoch 1/149 (batch 2/24) - categorical_accuracy: 0.45 Validation: Epoch 1/149 (batch 3/24) - categorical_accuracy: 0.2 Validation: Epoch 1/149 (batch 4/24) - categorical_accuracy: 0.875 Validation: Epoch 1/149 (batch 5/24) - categorical_accuracy: 0.575 Validation: Epoch 1/149 (batch 6/24) - categorical_accuracy: 0.5 Validation: Epoch 1/149 (batch 7/24) - categorical_accuracy: 0.575 Validation: Epoch 1/149 (batch 8/24) - categorical_accuracy: 0.45 Validation: Epoch 1/149 (batch 9/24) - categorical_accuracy: 0.525 Validation: Epoch 1/149 (batch 10/24) - categorical_accuracy: 0.975 Validation: Epoch 1/149 (batch 11/24) - categorical_accuracy: 0.875 Validation: Epoch 1/149 (batch 12/24) - categorical_accuracy: 0.775 Validation: Epoch 1/149 (batch 13/24) - categorical_accuracy: 0.875 Validation: Epoch 1/149 (batch 14/24) - categorical_accuracy: 0.9 Validation: Epoch 1/149 (batch 15/24) - categorical_accuracy: 0.775 Validation: Epoch 1/149 (batch 16/24) - categorical_accuracy: 0.925 Validation: Epoch 1/149 (batch 17/24) - categorical_accuracy: 0.925 Validation: Epoch 1/149 (batch 18/24) - categorical_accuracy: 0.95 Validation: Epoch 1/149 (batch 19/24) - categorical_accuracy: 0.875 Validation: Epoch 1/149 (batch 20/24) - categorical_accuracy: 0.8 Validation: Epoch 1/149 (batch 21/24) - categorical_accuracy: 0.7 Validation: Epoch 1/149 (batch 22/24) - categorical_accuracy: 0.425 Validation: Epoch 1/149 (batch 23/24) - categorical_accuracy: 0.7 Validation: Epoch 1/149 (batch 24/24) - categorical_accuracy: 0.65 Validation: Epoch 1/149 (batch 25/24) - categorical_accuracy: 0.15 <--- ???
Tip: use sequential LoadBatch for validation and DataGenerator for training.
LoadBatch
DataGenerator was removed with PR #19
DataGenerator sometimes skips some batches:
Or it saves them for later:
Tip: use sequential
LoadBatch
for validation and DataGenerator for training.