The claim in the readme.md that one can run all tasks in the evaluation suite using the asterisk is no longer true. This needs to be removed from the readme.
Results are not sent to a CSV, but to a JSON as specified in template_congif.yaml. Either a CSV feature needs to be implemented or this line needs to be deleted; either way, JSON needs to be added in as the saving format.
Latency benchmarking introduced in #63 is not documented.
While we're in the business of correcting the README, the heading Citations should be Citation, as it will only contain the citation for our own paper.
And the string quotations need to be deleted from the tasks parameter example here, with latency added, per our claim in the README that yaml doesn't expect strings.
The claim in the
readme.md
that one can run all tasks in the evaluation suite using the asterisk is no longer true. This needs to be removed from the readme.Results are not sent to a CSV, but to a JSON as specified in template_congif.yaml. Either a CSV feature needs to be implemented or this line needs to be deleted; either way, JSON needs to be added in as the saving format.
Latency benchmarking introduced in #63 is not documented.
While we're in the business of correcting the README, the heading
Citations
should beCitation
, as it will only contain the citation for our own paper.Overall a lot to work on here.