Closed napulen closed 3 years ago
The filtered (including validation) interval representation
Epoch 1/10
194/194 [==============================] - 8s 32ms/step - loss: 3.7976 - categorical_accuracy: 0.1476 - val_loss: 2.6604 - val_categorical_accuracy: 0.3047
Epoch 2/10
194/194 [==============================] - 6s 29ms/step - loss: 2.4368 - categorical_accuracy: 0.3671 - val_loss: 2.4694 - val_categorical_accuracy: 0.3586
Epoch 3/10
194/194 [==============================] - 6s 29ms/step - loss: 2.3088 - categorical_accuracy: 0.3876 - val_loss: 2.3679 - val_categorical_accuracy: 0.3765
Epoch 4/10
194/194 [==============================] - 6s 29ms/step - loss: 2.2002 - categorical_accuracy: 0.4115 - val_loss: 2.3182 - val_categorical_accuracy: 0.3956
Epoch 5/10
194/194 [==============================] - 6s 29ms/step - loss: 2.1624 - categorical_accuracy: 0.4207 - val_loss: 2.2839 - val_categorical_accuracy: 0.4031
Epoch 6/10
194/194 [==============================] - 6s 29ms/step - loss: 2.1479 - categorical_accuracy: 0.4262 - val_loss: 2.2577 - val_categorical_accuracy: 0.4060
Epoch 7/10
194/194 [==============================] - 6s 29ms/step - loss: 2.0991 - categorical_accuracy: 0.4362 - val_loss: 2.2471 - val_categorical_accuracy: 0.4066
Epoch 8/10
194/194 [==============================] - 6s 29ms/step - loss: 2.0593 - categorical_accuracy: 0.4416 - val_loss: 2.2297 - val_categorical_accuracy: 0.4130
Epoch 9/10
194/194 [==============================] - 6s 29ms/step - loss: 2.0680 - categorical_accuracy: 0.4394 - val_loss: 2.2189 - val_categorical_accuracy: 0.4090
Epoch 10/10
194/194 [==============================] - 6s 29ms/step - loss: 2.0387 - categorical_accuracy: 0.4467 - val_loss: 2.2102 - val_categorical_accuracy: 0.4148
It outperforms the micchi19
in predicting roman numeral labels.
The same experiment as before, except that only the training data has been filtered:
Epoch 1/10
194/194 [==============================] - 6s 24ms/step - loss: 3.7976 - categorical_accuracy: 0.1476 - val_loss: 2.6734 - val_categorical_accuracy: 0.3002
Epoch 2/10
194/194 [==============================] - 4s 22ms/step - loss: 2.4368 - categorical_accuracy: 0.3671 - val_loss: 2.4880 - val_categorical_accuracy: 0.3526
Epoch 3/10
194/194 [==============================] - 4s 22ms/step - loss: 2.3088 - categorical_accuracy: 0.3876 - val_loss: 2.3916 - val_categorical_accuracy: 0.3702
Epoch 4/10
194/194 [==============================] - 4s 22ms/step - loss: 2.2002 - categorical_accuracy: 0.4115 - val_loss: 2.3458 - val_categorical_accuracy: 0.3884
Epoch 5/10
194/194 [==============================] - 4s 22ms/step - loss: 2.1624 - categorical_accuracy: 0.4207 - val_loss: 2.3129 - val_categorical_accuracy: 0.3964
Epoch 6/10
194/194 [==============================] - 4s 22ms/step - loss: 2.1479 - categorical_accuracy: 0.4262 - val_loss: 2.2891 - val_categorical_accuracy: 0.3998
Epoch 7/10
194/194 [==============================] - 4s 21ms/step - loss: 2.0991 - categorical_accuracy: 0.4362 - val_loss: 2.2776 - val_categorical_accuracy: 0.4002
Epoch 8/10
194/194 [==============================] - 4s 21ms/step - loss: 2.0593 - categorical_accuracy: 0.4416 - val_loss: 2.2614 - val_categorical_accuracy: 0.4081
Epoch 9/10
194/194 [==============================] - 4s 21ms/step - loss: 2.0680 - categorical_accuracy: 0.4394 - val_loss: 2.2517 - val_categorical_accuracy: 0.4052
Epoch 10/10
194/194 [==============================] - 4s 21ms/step - loss: 2.0387 - categorical_accuracy: 0.4467 - val_loss: 2.2433 - val_categorical_accuracy: 0.4091
I just ran an experiment where I filter out all the data that has a bad quality score from my metrics:
This is the output
The performance is better than when I use all the data
This is good news. Less data of higher quality is better. Also, my metrics seem to work.
Big caveat on this experiment: I applied the filter to the validation portion too. In practice, I shouldn't filter the validation data in this way at all.
Trying out the filter in training-only data is going to be the next step.
Also, this was predicting roman numerals from the Micchi19 representation. I need to verify how well the intervallic input representation does.