xlang-ai / Spider2

Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
https://spider2-sql.github.io
Apache License 2.0
171 stars 14 forks source link

How to get access to BigQuery Database? Default credential seems expired. #10

Closed BugMaker-Boyan closed 1 month ago

BugMaker-Boyan commented 1 month ago

Hi, I try to use the scripts in spider2-lite/evaluation_suit, following your README instructions, but it does not work.

python evaluate.py --result_dir example_submission_folder

Output:

0%| | 0/3 [00:00<?, ?it/s]>>>Evaluating bq026_2... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 33%|###3 | 1/3 [00:00<00:00, 2.41it/s]>>>Evaluating bq117... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 67%|######6 | 2/3 [00:00<00:00, 2.45it/s]>>>Evaluating bq207... Error occurred while fetching data: ('invalid_grant: Invalid JWT Signature.', {'error': 'invalid_grant', 'error_description': 'Invalid JWT Signature.'}) 100%|##########| 3/3 [00:01<00:00, 2.43it/s] {'bq026_2': 0, 'bq117': 0, 'bq207': 0} Final score: 0.0 0.0

lfy79001 commented 1 month ago

Hi, please follow this guideline to get your own service account and the json file, replace the original bigquery_credential.json by yours.