Closed BugMaker-Boyan closed 1 month ago
Hi, I try to use the scripts in spider2-lite/evaluation_suit, following your README instructions, but it does not work.
python evaluate.py --result_dir example_submission_folder
Output:
0%| | 0/3 [00:00<?, ?it/s]>>>Evaluating bq026_2... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 33%|###3 | 1/3 [00:00<00:00, 2.41it/s]>>>Evaluating bq117... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 67%|######6 | 2/3 [00:00<00:00, 2.45it/s]>>>Evaluating bq207... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 100%|##########| 3/3 [00:01<00:00, 2.43it/s] {'bq026_2': 0, 'bq117': 0, 'bq207': 0} Final score: 0.0 0.0
Hi, please follow this guideline to get your own service account and the json file, replace the original bigquery_credential.json by yours.
bigquery_credential.json
Hi, I try to use the scripts in spider2-lite/evaluation_suit, following your README instructions, but it does not work.
python evaluate.py --result_dir example_submission_folder
Output:
0%| | 0/3 [00:00<?, ?it/s]>>>Evaluating bq026_2... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 33%|###3 | 1/3 [00:00<00:00, 2.41it/s]>>>Evaluating bq117... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 67%|######6 | 2/3 [00:00<00:00, 2.45it/s]>>>Evaluating bq207... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 100%|##########| 3/3 [00:01<00:00, 2.43it/s] {'bq026_2': 0, 'bq117': 0, 'bq207': 0} Final score: 0.0 0.0