sodadata / soda-core

:zap: Data quality testing for the modern data stack (SQL, Spark, and Pandas) https://www.soda.io
https://go.soda.io/core-docs
Apache License 2.0
1.89k stars 208 forks source link

automated monitoring fails when wildcards are used (i.e. 'include %') #1625

Open geertvanzoest opened 2 years ago

geertvanzoest commented 2 years ago

Problem When automated monitoring (include %) is applied to a sqlserver-source with object names containing whitespace(s), the following syntax error is thrown:

Query execution error in nav.Tax Jurisdiction Translation.aggregation[0]: ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Incorrect syntax near 'Translation'. (102) (SQLExecDirectW)")
SELECT 
  COUNT(*) 
FROM dbo.Tax Jurisdiction Translation
  | ('42000', "[42000] [Microsoft][ODBC Driver 18 for SQL Server][SQL Server]Incorrect syntax near 'Translation'. (102) (SQLExecDirectW)")

Suspected cause This happens because of omitted object identifiers in the SQL-statements Soda generates. MSSQL expects dbo.Tax Jurisdiction Translation to be enclosed with (preferably) brackets [dbo].[Tax Jurisdiction Translation]. Seen this issue before in Soda <-> MSSQL (https://github.com/sodadata/soda-sql/issues/171, https://github.com/sodadata/soda-sql/issues/181).

Steps to reproduce

  1. Have a Microsoft SQL Server instance with a table containing a column with a white-space character in it's name.
  2. Configure checks.yml with at least:
    automated monitoring:
    datasets:
    - include %
  3. Run soda scan

System OS: Windows Server 2022, Windows 10 Enterprise 21H2 Docker version: 20.10.17 Soda Core version: 3.0.10 Data source type: Microsoft SQL Server 2019 (15.0.2080.9)

Let me know if I can test anything for you on sqlserver! ✋🏼

jmarien commented 2 years ago

SODA-1209