msaisumanth
commented
5 years ago I think your model is just overfitting. I believe 500 examples is quite small for a problem like this.
If you don't have access to more data, you may try reducing the size of the encoder.
Here's a sample model_definition file you can refer to:
input_features:
-
name: review_text
type: text
encoder: parallel_cnn
embedding_size: 28
filter_size: 1
num_filters: 28
level: word
output_features:
-
name: rating
type: category
training:
epochs: 5
learning_rate: 0.1
combiner:
type: concat
Using pre-trained embeddings also might help, given you have very few examples.
w4nderlust
pdsing
I am trying to build a review to rating prediction model. I have a CSV of reviews and corresponding ratings by the users data_500.csv.txt.
I am using Google Colab.
I create a model using: !ludwig train --data_csv data_500.csv --model_definition "{input_features: [{name: review_text, type: text}], output_features: [{name: rating, type: category}]}". The logs are attached below.
When I try to predict the rating on my test data (test_20.csv.txt), I get all ratings as 5.
I am using !ludwig predict --data_csv test_20.csv --model_path results/experiment_run_0/model for prediction. The log is attached below.
I think my model is not getting trained at all. Can you please guide me where I am going wrong?
When I train, I get the following log -
ludwig_version: '0.1.0' command: ('/usr/local/bin/ludwig train --data_csv data_500.csv --model_definition ' '{input_features: [{name: review_text, type: text}], output_features: [{name: ' 'rating, type: category}]}') dataset_type: 'data_500.csv' model_definition: { 'combiner': {'type': 'concat'}, 'input_features': [ { 'encoder': 'parallel_cnn', 'level': 'word', 'name': 'review_text', 'tied_weights': None, 'type': 'text'}], 'output_features': [ { 'dependencies': [], 'loss': { 'class_distance_temperature': 0, 'class_weights': 1, 'confidence_penalty': 0, 'distortion': 1, 'labels_smoothing': 0, 'negative_samples': 0, 'robust_lambda': 0, 'sampler': None, 'type': 'softmax_cross_entropy', 'unique': False, 'weight': 1}, 'name': 'rating', 'reduce_dependencies': 'sum', 'reduce_input': 'sum', 'top_k': 3, 'type': 'category'}], 'preprocessing': { 'bag': { 'fill_value': '', 'format': 'space', 'lowercase': 10000, 'missing_value_strategy': 'fill_with_const', 'most_common': False}, 'binary': { 'fill_value': 0, 'missing_value_strategy': 'fill_with_const'}, 'category': { 'fill_value': '',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 10000},
'force_split': False,
'image': {'missing_value_strategy': 'backfill'},
'numerical': { 'fill_value': 0,
'missing_value_strategy': 'fill_with_const'},
'sequence': { 'fill_value': '',
'format': 'space',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 20000,
'padding': 'right',
'padding_symbol': '',
'sequence_length_limit': 256,
'unknown_symbol': ''},
'set': { 'fill_value': '',
'format': 'space',
'lowercase': False,
'missing_value_strategy': 'fill_with_const',
'most_common': 10000},
'split_probabilities': (0.7, 0.1, 0.2),
'stratify': None,
'text': { 'char_format': 'characters',
'char_most_common': 70,
'char_sequence_length_limit': 1024,
'fill_value': '',
'lowercase': True,
'missing_value_strategy': 'fill_with_const',
'padding': 'right',
'padding_symbol': '',
'unknown_symbol': '',
'word_format': 'space_punct',
'word_most_common': 20000,
'word_sequence_length_limit': 256},
'timeseries': { 'fill_value': '',
'format': 'space',
'missing_value_strategy': 'fill_with_const',
'padding': 'right',
'padding_value': 0,
'timeseries_length_limit': 256}},
'training': { 'batch_size': 128,
'bucketing_field': None,
'decay': False,
'decay_rate': 0.96,
'decay_steps': 10000,
'dropout_rate': 0.0,
'early_stop': 5,
'epochs': 100,
'gradient_clipping': None,
'increase_batch_size_on_plateau': 0,
'increase_batch_size_on_plateau_max': 512,
'increase_batch_size_on_plateau_patience': 5,
'increase_batch_size_on_plateau_rate': 2,
'learning_rate': 0.001,
'learning_rate_warmup_epochs': 5,
'optimizer': { 'beta1': 0.9,
'beta2': 0.999,
'epsilon': 1e-08,
'type': 'adam'},
'reduce_learning_rate_on_plateau': 0,
'reduce_learning_rate_on_plateau_patience': 5,
'reduce_learning_rate_on_plateau_rate': 0.5,
'regularization_lambda': 0,
'regularizer': 'l2',
'staircase': False,
'validation_field': 'combined',
'validation_measure': 'loss'}}
Using full raw csv, no hdf5 and json file with the same name have been found Building dataset (it may take a while) Loading NLP pipeline Writing dataset Writing train set metadata with vocabulary Training set: 346 Validation set: 48 Test set: 106 WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/losses/losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/losses/losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide.
╒══════════╕ │ TRAINING │ ╘══════════╛
2019-02-25 07:53:20.563543: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz 2019-02-25 07:53:20.567917: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x21cf860 executing computations on platform Host. Devices: 2019-02-25 07:53:20.567964: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0):,
Epoch 1 Training: 100% 3/3 [00:10<00:00, 3.79s/it] Evaluation train: 100% 3/3 [00:02<00:00, 1.02s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.40it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.10it/s] Took 15.2982s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 4.8280 │ 0.1301 │ 0.9191 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 4.1219 │ 0.2917 │ 0.8958 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 4.5526 │ 0.1698 │ 0.9528 │ ╘══════════╧════════╧════════════╧═════════════╛ Validation loss on combined improved, model saved
Epoch 2 Training: 100% 3/3 [00:10<00:00, 3.45s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.02s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.34it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.10it/s] Took 14.6133s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 1.6862 │ 0.7543 │ 0.8410 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 2.9535 │ 0.5833 │ 0.6875 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 1.9904 │ 0.7264 │ 0.7925 │ ╘══════════╧════════╧════════════╧═════════════╛ Validation loss on combined improved, model saved
Epoch 3 Training: 100% 3/3 [00:10<00:00, 3.48s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.01s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.34it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.08it/s] Took 14.6806s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 1.2675 │ 0.7543 │ 0.8410 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 2.2575 │ 0.5833 │ 0.6875 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 1.4940 │ 0.7264 │ 0.7925 │ ╘══════════╧════════╧════════════╧═════════════╛ Validation loss on combined improved, model saved
Epoch 4 Training: 100% 3/3 [00:10<00:00, 3.46s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.01s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.37it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.08it/s] Took 14.6354s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 1.1529 │ 0.3988 │ 0.9133 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.2190 │ 0.2708 │ 0.8958 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 1.1510 │ 0.3585 │ 0.9340 │ ╘══════════╧════════╧════════════╧═════════════╛ Validation loss on combined improved, model saved
Epoch 5 Training: 100% 3/3 [00:10<00:00, 3.48s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.01s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.37it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.08it/s] Took 14.7180s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 1.0847 │ 0.7543 │ 0.9364 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.6780 │ 0.5833 │ 0.9583 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 1.2223 │ 0.7264 │ 0.9057 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 1 epoch ago
Epoch 6 Training: 100% 3/3 [00:10<00:00, 3.45s/it] Evaluation train: 100% 3/3 [00:02<00:00, 1.01s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.34it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.09it/s] Took 14.5831s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.9503 │ 0.7543 │ 0.9364 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.1677 │ 0.5833 │ 0.9583 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 1.0606 │ 0.7264 │ 0.9057 │ ╘══════════╧════════╧════════════╧═════════════╛ Validation loss on combined improved, model saved
Epoch 7 Training: 100% 3/3 [00:10<00:00, 3.47s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.02s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.36it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.09it/s] Took 14.6443s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.9198 │ 0.7543 │ 0.9277 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.4745 │ 0.5833 │ 0.8958 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 0.9475 │ 0.7264 │ 0.9528 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 1 epoch ago
Epoch 8 Training: 100% 3/3 [00:10<00:00, 3.43s/it] Evaluation train: 100% 3/3 [00:02<00:00, 1.00s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.38it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.10it/s] Took 14.4598s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.8892 │ 0.7543 │ 0.9133 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.2088 │ 0.5833 │ 0.8958 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 0.8997 │ 0.7264 │ 0.9340 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 2 epochs ago
Epoch 9 Training: 100% 3/3 [00:10<00:00, 3.45s/it] Evaluation train: 100% 3/3 [00:02<00:00, 1.01s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.37it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.09it/s] Took 14.5781s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.8413 │ 0.7543 │ 0.9364 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.2812 │ 0.5833 │ 0.9583 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 0.9747 │ 0.7264 │ 0.9057 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 3 epochs ago
Epoch 10 Training: 100% 3/3 [00:10<00:00, 3.45s/it] Evaluation train: 100% 3/3 [00:02<00:00, 1.01it/s] Evaluation vali : 100% 1/1 [00:00<00:00, 2.40it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.09it/s] Took 14.5225s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.8365 │ 0.7543 │ 0.9364 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.2758 │ 0.5833 │ 0.9583 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 0.9576 │ 0.7264 │ 0.9057 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 4 epochs ago
Epoch 11 Training: 100% 3/3 [00:10<00:00, 3.47s/it] Evaluation train: 100% 3/3 [00:03<00:00, 1.02s/it] Evaluation vali : 100% 1/1 [00:00<00:00, 2.41it/s] Evaluation test : 100% 1/1 [00:00<00:00, 1.09it/s] Took 14.6348s ╒══════════╤════════╤════════════╤═════════════╕ │ rating │ loss │ accuracy │ hits_at_k │ ╞══════════╪════════╪════════════╪═════════════╡ │ train │ 0.8353 │ 0.7543 │ 0.9277 │ ├──────────┼────────┼────────────┼─────────────┤ │ vali │ 1.2154 │ 0.5833 │ 0.8958 │ ├──────────┼────────┼────────────┼─────────────┤ │ test │ 0.8837 │ 0.7264 │ 0.9528 │ ╘══════════╧════════╧════════════╧═════════════╛ Last improvement of loss on combined happened 5 epochs ago
EARLY STOPPING due to lack of validation improvement, it has been 5 epochs since last validation accuracy improvement
Best validation model epoch: Best validation model loss on validation set combined: 4.121904691060384 Best validation model loss on test set combined: 4.5525981975051595
Finished: experiment_run Saved to: results/experiment_run_0
Log at the time of prediction
| | | |_ () | | || /
\ V V / / _| ||_,_,|_/_/|_, | |_/ ludwig v0.1.0 - PredictDataset type: generic Dataset path: test_20.csv Model path: results/experiment_run_0/model Output path: results_0
Loading metadata from: results/experiment_run_0/model/train_set_metadata.json
╒═══════════════╕ │ LOADING MODEL │ ╘═══════════════╛
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/losses/losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/losses/losses_impl.py:209: to_float (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.cast instead. WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide. From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:102: div (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version. Instructions for updating: Deprecated in favor of operator or tf.math.divide.
╒═════════╕ │ PREDICT │ ╘═════════╛
2019-02-25 08:49:16.742020: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2300000000 Hz 2019-02-25 08:49:16.742365: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x228a7e0 executing computations on platform Host. Devices: 2019-02-25 08:49:16.742433: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0):,
WARNING:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
From /usr/local/lib/python3.6/dist-packages/tensorflow/python/training/saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.
Instructions for updating:
Use standard file APIs to check for files with this prefix.
INFO:tensorflow:Restoring parameters from results/experiment_run_0/model/model_weights
Restoring parameters from results/experiment_run_0/model/model_weights
Evaluation: 100% 1/1 [00:00<00:00, 4.01it/s]
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples.
'precision', 'predicted', average, warn_for)
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1145: UndefinedMetricWarning: Recall is ill-defined and being set to 0.0 in labels with no true samples.
'recall', 'true', average, warn_for)
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1143: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no predicted samples.
'precision', 'predicted', average, warn_for)
/usr/local/lib/python3.6/dist-packages/sklearn/metrics/classification.py:1145: UndefinedMetricWarning: F-score is ill-defined and being set to 0.0 in labels with no true samples.
'recall', 'true', average, warn_for)
===== rating ===== accuracy: 0.0 hits_at_k: 0.0 loss: 17.951489535245027 overall_stats: { 'avg_f1_score_macro': 0.0, 'avg_f1_score_micro': 0.0, 'avg_f1_score_weighted': 0.0, 'avg_precision_macro': 0.0, 'avg_precision_micro': 0.0, 'avg_precision_weighted': 0.0, 'avg_recall_macro': 0.0, 'avg_recall_micro': 0.0, 'avg_recall_weighted': 0.0, 'kappa_score': 0.0, 'overall_accuracy': 0.0} per_class_stats: {: { 'accuracy': 0.0,
'f1_score': 0,
'fall_out': 1.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 0,
'false_omission_rate': 1.0,
'false_positive_rate': 1.0,
'false_positives': 22,
'hit_rate': 0,
'informedness': -1.0,
'markedness': -1.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 0,
'positive_predictive_value': 0.0,
'precision': 0.0,
'recall': 0,
'sensitivity': 0,
'specificity': 0.0,
'true_negative_rate': 0.0,
'true_negatives': 0,
'true_positive_rate': 0,
'true_positives': 0},
label_5: { 'accuracy': 0.0,
'f1_score': 0,
'fall_out': 1.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 22,
'false_omission_rate': 1.0,
'false_positive_rate': 1.0,
'false_positives': 0,
'hit_rate': 0.0,
'informedness': -1.0,
'markedness': -1.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 0.0,
'positive_predictive_value': 0,
'precision': 0,
'recall': 0.0,
'sensitivity': 0.0,
'specificity': 0,
'true_negative_rate': 0,
'true_negatives': 0,
'true_positive_rate': 0.0,
'true_positives': 0},
label_4: { 'accuracy': 1.0,
'f1_score': 0,
'fall_out': 0.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 0,
'false_omission_rate': 0.0,
'false_positive_rate': 0.0,
'false_positives': 0,
'hit_rate': 0,
'informedness': 0.0,
'markedness': 0.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 1.0,
'positive_predictive_value': 0,
'precision': 0,
'recall': 0,
'sensitivity': 0,
'specificity': 1.0,
'true_negative_rate': 1.0,
'true_negatives': 22,
'true_positive_rate': 0,
'true_positives': 0},
label_1: { 'accuracy': 1.0,
'f1_score': 0,
'fall_out': 0.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 0,
'false_omission_rate': 0.0,
'false_positive_rate': 0.0,
'false_positives': 0,
'hit_rate': 0,
'informedness': 0.0,
'markedness': 0.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 1.0,
'positive_predictive_value': 0,
'precision': 0,
'recall': 0,
'sensitivity': 0,
'specificity': 1.0,
'true_negative_rate': 1.0,
'true_negatives': 22,
'true_positive_rate': 0,
'true_positives': 0},
label_2: { 'accuracy': 1.0,
'f1_score': 0,
'fall_out': 0.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 0,
'false_omission_rate': 0.0,
'false_positive_rate': 0.0,
'false_positives': 0,
'hit_rate': 0,
'informedness': 0.0,
'markedness': 0.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 1.0,
'positive_predictive_value': 0,
'precision': 0,
'recall': 0,
'sensitivity': 0,
'specificity': 1.0,
'true_negative_rate': 1.0,
'true_negatives': 22,
'true_positive_rate': 0,
'true_positives': 0},
label_3: { 'accuracy': 1.0,
'f1_score': 0,
'fall_out': 0.0,
'false_discovery_rate': 1.0,
'false_negative_rate': 1.0,
'false_negatives': 0,
'false_omission_rate': 0.0,
'false_positive_rate': 0.0,
'false_positives': 0,
'hit_rate': 0,
'informedness': 0.0,
'markedness': 0.0,
'matthews_correlation_coefficient': 0,
'miss_rate': 1.0,
'negative_predictive_value': 1.0,
'positive_predictive_value': 0,
'precision': 0,
'recall': 0,
'sensitivity': 0,
'specificity': 1.0,
'true_negative_rate': 1.0,
'true_negatives': 22,
'true_positive_rate': 0,
'true_positives': 0}}
Saved to: results_0