search

xlang-ai / Spider2

Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows
https://spider2-sql.github.io
Apache License 2.0
147 stars 12 forks source link

example evaluation queries failing [spider2-lite] #14

Open snash4 opened 3 weeks ago

snash4 commented 3 weeks ago

Hi, Two queries out of 3 in example_submission_folder are failing. Query bq026_2 return this error: Error occurred while fetching data: 403 Access Denied: Table spider2-public-data:patents.publications: User does not have permission to query table spider2-public-data:patents.publications, or perhaps it does not exist.; reason: accessDenied, message: Access Denied: Table spider2-public-data:patents.publications: User does not have permission to query table spider2-public-data:patents.publications, or perhaps it does not exist.

Query bq207 error: Error occurred while fetching data: 403 Access Denied: Table spider2-public-data:patents.publications: User does not have permission to query table spider2-public-data:patents.publications, or perhaps it does not exist.; reason: accessDenied, message: Access Denied: Table spider2-public-data:patents.publications: User does not have permission to query table spider2-public-data:patents.publications, or perhaps it does not exist.

lfy79001 commented 3 weeks ago

please add your service account in this form

snash4 commented 3 weeks ago

thanks for access. I can access the databases. However, evaluating example_folder queries. I get {'bq026_2': 0, 'bq117': 1, 'bq207': 0} Is this the expected result? Only one successful query (bbq117) from the examples.

thanks.