search

neulab / explainaboard_client

1 stars 1 forks source link

Bug?: explainaboard_api_client.exceptions.ApiTypeError #26

Closed pfliu-nlp closed 2 years ago

pfliu-nlp commented 2 years ago

When evaluating named datasets from DataLab,

python -m explainaboard_client.cli.upload_system \
  --email XXX --api_key YYY \
  --task named-entity-recognition \
  --system_name test_ner \
  --system_output conll2003-elmo-output.conll --output_file_type conll \
  --dataset conll2003 --sub_dataset ner --split test \
  --source_language en --target_language en \
  --public

Errors:

File "/usr1/data/pliu3/neulab/ExplainaBoard-Walkthrough/roles/instructors/fudan_nlp/venv/lib/python3.9/site-packages/explainaboard_api_client/model_utils.py", line 1582, in validate_and_convert_types
    raise get_type_error(input_value, path_to_item, valid_classes,
explainaboard_api_client.exceptions.ApiTypeError: Invalid type for variable 'dataset_metadata_id'. Required value type is str and passed type was tuple at ['dataset_metadata_id']
pfliu-nlp commented 2 years ago

https://github.com/neulab/explainaboard_client/blob/8ad61b289bb97bc34f3b303db9b4f04fa7e3dd04/explainaboard_client/cli/evaluate_system.py#L164

It should be 

        metadata.dataset_metadata_id = generate_dataset_id(args.dataset, args.sub_dataset)

!

pfliu-nlp commented 2 years ago

fixed: https://github.com/neulab/explainaboard_client/pull/30