google / zetasql

ZetaSQL - Analyzer Framework for SQL
Apache License 2.0
2.32k stars 219 forks source link

BigQuery _TABLE_SUFFIX support #88

Open cthrysoe opened 2 years ago

cthrysoe commented 2 years ago

Hi, In BigQuery, tables can be referenced by a wildcard:

select * from `dataset.table` where _TABLE_SUFFIX = 'suffix'

Docs

When analyzing a query, that references a wildcard table through _TABLE_SUFFIX, I get the error:

com.google.zetasql.SqlException: Unrecognized name: _TABLE_SUFFIX

Can I add support for this somehow? Thanks

Matts966 commented 2 years ago

Example implementation:

https://github.com/Matts966/alphasql/blob/master/samples/sample-ci/python_entrypoint.py https://github.com/Matts966/alphasql/blob/master/alphasql/json_schema_reader.h

cthrysoe commented 2 years ago

Thanks @Matts966 , So if I understand your code correctly, you are adding the _TABLE_SUFFIX as a column to the table that you then add to your catalog. That allows me to analyze the query, do you know how I can get a reference to the referred table(s) behind the wildcard, for example in this statement:

select * from `dataset.table*` where _TABLE_SUFFIX = '20220101'

I want to be able to identify that the query hits the table dataset.table20220101 and similar if I were to specify a range in the table suffix. Do you know if this is possible to obtain? Thanks

Matts966 commented 2 years ago

I feel it is difficult because string values can be modified dynamically, e.g. _TABLE_SUFFIX = a + b. But you can obtain concrete values if it is static literal, by reading resolved AST.