Open bschulth opened 1 month ago
Here is a reprex on spark showing correct SQL works and current SQL does not.
con <- DBI::dbConnect(
odbc::databricks(),
httpPath = Sys.getenv("DATABRICKS_HTTPPATH"),
useNativeQuery = FALSE,
bigint = "numeric"
)
DBI::dbGetQuery(con,
"select observation_period_start_date, ADD_MONTHS('`observation_period_start_date`', 1.0 * 12.0) as day
from gibleed.observation_period limit 5")
#> observation_period_start_date day
#> 1 1963-12-31 <NA>
#> 2 2009-04-26 <NA>
#> 3 2002-01-30 <NA>
#> 4 1971-10-14 <NA>
#> 5 2009-05-30 <NA>
DBI::dbGetQuery(con,
"select observation_period_start_date, ADD_MONTHS(`observation_period_start_date`, 1.0 * 12.0) as day
from gibleed.observation_period limit 5")
#> observation_period_start_date day
#> 1 1963-12-31 1964-12-31
#> 2 2009-04-26 2010-04-26
#> 3 2002-01-30 2003-01-30
#> 4 1971-10-14 1972-10-14
#> 5 2009-05-30 2010-05-30
DBI::dbDisconnect(con)
Created on 2024-06-05 with reprex v2.1.0
When translating sql on a
Spark SQL
connection, the 'add_year' translation is double quoting the column name:This fails on Spark. I would expect single quotes like:
This seems to be coming from the triple-bang expression here
!!!x
: https://github.com/tidyverse/dbplyr/blob/8f2fcb0852994afe11cc6792ef59ef5ffda84936/R/backend-spark-sql.R#L47There even seems to be a bad unit test for this here:
https://github.com/tidyverse/dbplyr/blob/8f2fcb0852994afe11cc6792ef59ef5ffda84936/tests/testthat/test-backend-spark-sql.R#L3C35-L3C50
If I change the triple-bang
!!!x
to a double-bang!!x
:it seems to produce the correct output.
Here is the reprex: