I submitted the docker image of the example date annotator hosted on NLPSandbox.io as a first step to troubleshoot an issue encountered by a participant in AMIA Informatics Summit. I received the scores by email for the i2b2 and MCW dataset, but not for the Mayo Clinic dataset.
Scores on i2b2 dataset:
Your submission (id: 9719957) is scored, below are your results for i2b2-phi-20210622:
location_F1_instance_relax : 0.25
location_precision_instance_relax : 0.26
location_recall_instance_relax : 0.25
location_F1_instance_strict : 0.25
location_precision_instance_strict : 0.26
location_recall_instance_strict : 0.25
location_F1_token_strict : 0.34
location_precision_token_strict : 0.37
location_recall_token_strict : 0.31
Scores on MCW dataset:
Your submission (id: 9719957) is scored, below are your results for mayo-clinic-20211026:
location_F1_instance_relax : 0.3
location_precision_instance_relax : 0.25
location_recall_instance_relax : 0.39
location_F1_instance_strict : 0.3
location_precision_instance_strict : 0.25
location_recall_instance_strict : 0.39
location_F1_token_strict : 0.3
location_precision_token_strict : 0.25
location_recall_token_strict : 0.39
I submitted the docker image of the example date annotator hosted on NLPSandbox.io as a first step to troubleshoot an issue encountered by a participant in AMIA Informatics Summit. I received the scores by email for the i2b2 and MCW dataset, but not for the Mayo Clinic dataset.
Scores on i2b2 dataset:
Scores on MCW dataset: