search

sodadata / soda-sql

Soda SQL and Soda Spark have been deprecated and replaced by Soda Core. docs.soda.io/soda-core/overview.html
https://docs.soda.io/
Apache License 2.0
60 stars 16 forks source link

Spark dialect: describe table returns partitioning column for delta tables #121

Closed JCZuurmond closed 2 years ago

JCZuurmond commented 2 years ago

Describe the bug For delta tables DESCRIBE TABLE returns an empty column, # Partitioning and all partitioning columns. These columns show up in Soda cloud as if these are "normal" columns.

To Reproduce Steps to reproduce the behavior:

  1. Run soda scan ... against a delta table

Context It probably depends on the table type (what comes after USING). I see this for a delta table, it might also happen with other table types.

Python Version: 3.7

Soda SQL Version: 2.1.0b17

Warehouse Type: Spark dialect