test

[yuppx]

I want to make an example to test this model, what should I do?

[bmccann]

Do you want to try a single (context, question, answer) triplet? Or do you want to add an entire test dataset?

[yuppx]

a single (context, question, answer) triplet。

[bmccann]

This can maybe be of some help: https://github.com/salesforce/decaNLP/commit/0e33e91dcadd7a6bf2fa550d9083a6a9bc65ae37.

Inference on a Custom Dataset

Using a pretrained model or a model you have trained yourself, you can run on new, custom datasets easily by following the instructions below. In this example, we use the checkpoint for the best MQAN trained on the entirety of decaNLP (see the section on Pretrained Models to see how to get this checkpoint) to run on my_custom_dataset.

mkdir .data/my_custom_dataset/ touch .datda/my_custom_dataset/val.jsonl

TODO add examples line by line to val.jsonl in the form of a JSON dict: {"context": "The answer is answer.", "question": "What is the answer?", "answer": "answer"}

nvidia-docker run -it --rm -v pwd:/decaNLP/ decanlp bash -c "python /decaNLP/predict.py --evaluate valid --path /decaNLP/mqan_decanlp_qa_first --checkpoint_name model.pth --gpu 0 --tasks my_custom_dataset"