To evaluate the model on helios, the evaluator script must be used with the input and target samples.
python evaluator.py --input-file-path {input-file-path} --target-file-path {target-file-path}
For more information, see python evaluator.py --help
.
Note that the dependencies are in requirements.txt
.
They can be install with pip
.
pip install -r requirements.txt
All experiments can be reproduced with the scripts in the folders experiments
.
Custom experiments can be made with the help of the script run_experiment.py
.
For more information, see python run_experiment.py --help
.
Artifacts are stored in different locations :
logging/{model-name}/{datetime}/experiment.log
,models/{model-name}-{id}/{epoch}.*
,results/{model}-{id}/{datetime}/valid-{epoch}
,results/{model}-{id}/{datetime}/train-{epoch}
,results/{model}-{id}/{datetime}/history-{epoch}
.python generate_graphs.py --history_path='results/lstm_luong_attention/2020-04-01 21:58:21/history-17' output_path='results/lstm_luong_attention/2020-04-01 21:58:21/graphs'
Testing a masked language model can be done using the run_test_mlm.py script.
Example: python run_test_mlm.py --checkpoint 2 --message "father help <mask>me pick apples"
Predicted token: ed