search

paul-tqh-nguyen / reuters_topic_labelling

Deep learning to automatically label Reuter's articles with the relevant topics.
1 stars 0 forks source link

Examine efficacy of older hyperparameter selection #21

Closed paul-tqh-nguyen closed 4 years ago

paul-tqh-nguyen commented 4 years ago 
{"best_valid_loss": 0.013681062969185705,
 "number_of_epochs": 40,
 "most_recently_completed_epoch_index": 6,
 "batch_size": 1,
 "max_vocab_size": 25000,
 "vocab_size": 19882,
 "pre_trained_embedding_specification": "fasttext.simple.300d",
 "encoding_hidden_size": 512,
 "number_of_encoding_layers": 1,
 "attention_intermediate_size": 32,
 "number_of_attention_heads": 2,
 "dropout_probability": 0.25,
 "output_size": 119,
 "train_portion": 0.5,
 "validation_portion": 0.2,
 "testing_portion": 0.3,
 "number_of_parameters": 9575441,
 "test_f1": 0.8325288468240255,
 "test_loss": 0.012920586172941688}

https://github.com/paul-tqh-nguyen/reuters_topic_labelling/commit/c26143dc862513f961687dc4e040a13d54b2e14b beat this old score by a feature improvement. However, this old score had a lower loss. Would this hyperparameter set get better performance with the new feature? Let's investigate that.

paul-tqh-nguyen commented 4 years ago

Seeing the tragic discoveries from https://github.com/paul-tqh-nguyen/reuters_topic_labelling/commit/a842a476dc283e0e9cba8fc13526ee756d4c280b, I'll say that this issue is now unfortunately pointless.