Documentation Enhancement

[gkowalski]

The documentation in the README.md states

Evaluate the performance of a local prediction file:

docker run --rm nlpsandbox/cli evaluate

But does not explain the other options which are required , nor tell how to first get / produce these files

docker run --rm nlpsandbox/cli evaluate prediction --help

Usage: nlp-cli evaluate prediction [OPTIONS]

  Evaluate the performance of a local prediction file

Options:
  --pred_filepath PATH            Prediction filepath  [required]
  --gold_filepath PATH            Gold standard filepath  [required]
  --output PATH                   Specify output json path
  --eval_type [date|person|address]
                                  Type of evaluation.
  --help                          Show this message and exit.

[thomasyu888]

@tschaffter , I can add documentation that leads to the example prediction and goldstandard files, but I think the docs of obtaining these prediction / goldstandard files should be internal. For example:

1. Obtain predictions from a particular submission id (example: 3):

   nlp-cli community list-annotations --dataset_id {datasetid} --annotation_store_id submsision-3 --output prediction.json

2. Obtain goldstandard from a particular submission id (example: 3):

   nlp-cli community list-annotations --dataset_id {datasetid} --annotation_store_id goldstandard --output goldstandard.json

3. Evaluate

   nlp-cli evaluate prediction --pred_filepath prediction.json --gold_filepath goldstandard.json --eval_type date

[thomasyu888]

Opened a ticket here to track: https://github.com/nlpsandbox/nlpsandbox-website-synapse/issues/11

[github-actions[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.