I have followed the instructions to run the checkpoint model, and been able to successfully run it. However, I get different "answer"s on the demo and the trained model checkpoint.
Demo:
Passage: The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.
Question: Who was the injured?
Answer: cancer victim
Trained Model Checkpoint:
input.jsonl :
{"passage": "The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.", "question":"Who was the injured?"}
output.jsonl:
{"question": "Who was the injured?", "passage": "The family of a cancer victim that had won a $4.4 million judgment against Laboratory Corp. of America over a botched Pap smear test settled with the company Monday just before the parties were set to start a new trial on noneconomic damages in Florida federal court.", "predicted_ans": "against Laboratory Corp.", "answer": "against Laboratory Corp.", "inputs": [{"name": "question", "tokens": ["Who", "was", "the", "injured", "?"]}, {"name": "passage", "tokens": ["The", "family", "of", "a", "cancer", "victim", "that", "had", "won", "a", "$", "4.4", "million", "judgment", "against", "Laboratory", "Corp.", "of", "America", "over", "a", "botched", "Pap", "smear", "test", "settled", "with", "the", "company", "Monday", "just", "before", "the", "parties", "were", "set", "to", "start", "a", "new", "trial", "on", "noneconomic", "damages", "in", "Florida", "federal", "court", "."]}, {"name": "numbers", "tokens": ["4.4", "0", "100.0"]}, {"name": "dates", "tokens": ["-1/-1/-1"]}, {"name": "composed_numbers", "tokens": ["0.0", "8.8", "95.6", "104.4", "200.0"]}, {"name": "year_diffs", "tokens": ["0"]}, {"name": "count", "tokens": ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9"]}], "program_nested_expression": [{"name": "span", "identifier": 4}, [{"name": "relocate", "identifier": 3}, [{"name": "filter", "identifier": 2}, {"name": "find", "identifier": 1}]]], "program_lisp": "(span (relocate (filter find)))", "program_execution": []}
I have followed the instructions to run the checkpoint model, and been able to successfully run it. However, I get different "answer"s on the demo and the trained model checkpoint.
Demo:
Trained Model Checkpoint:
==> cancer victim vs. against Laboratory Corp