search

neulab / explainaboard_client

1 stars 1 forks source link

Evaluate system file simplified API #32

Closed neubig closed 2 years ago

neubig commented 2 years ago

Starts to address https://github.com/neulab/explainaboard_client/issues/31

This simplifies the API to evaluate systems to be a single API call such as the following:

result: dict = client.evaluate_system_file(
    system_output_file="sst2-output.txt",
    system_output_file_type="text",
    task="text-classification",
    system_name="test_cli",
    metric_names=["Accuracy"],
    source_language="en",
    target_language="en",
    dataset="sst2",
    split="test",
    shared_users=["explainaboard@gmail.com"],
)