eakmanrq / sqlframe

Turning PySpark Into a Universal DataFrame API
https://sqlframe.readthedocs.io/en/stable/
MIT License
290 stars 9 forks source link

feat: add printSchema support for duckdb and postgres #29

Closed eakmanrq closed 4 months ago

eakmanrq commented 4 months ago

Adds the ability to do "printSchema" in both DuckDB and Postgres.

The implementation works with engines that support temp views because they allow us to create a temp view and then get the view's data type from the information schema. BigQuery is not supported because it doesn't support temp views. A workaround is to create a temp table with limit 0/False like what we do in SQLMesh. The dataset it is created in is hidden and therefore will require an API call to get the column information. So a bit more involved so looking to do that later.

Issue: https://github.com/eakmanrq/sqlframe/issues/25