r-dbi / odbc

Connect to ODBC databases (using the DBI interface)
https://odbc.r-dbi.org/
Other
392 stars 107 forks source link

Encoding of Azure Data Warehouse secret key leads to base-64 error #410

Open alastairmatheson opened 3 years ago

alastairmatheson commented 3 years ago

Issue Description and Expected Result

Using dbExecute to run a query that contains a storage account key fails with Error: nanodbc/nanodbc.cpp:1617: 42000: [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

Running the same thing using RODBC's sqlQuery is successful. Running the same SQL query in SQL Server Management Studio also works.

My guess is it has to do with the way the characters in the storage account key are encoded before being send to the data warehouse, especially the / and + characters, but I am not sure. If that is the issue, I am also not sure how to change how the key is encoded.

Database

Azure Data Warehouse/Synapse Analytics

Reproducible Example

Difficult to give a reproducible example because of the specifics but here is the code I'm running:

library(odbc)
library(DBI)
library(keyring)

con <- DBI::dbConnect(odbc::odbc(),
                       driver = "ODBC Driver 17 for SQL Server",
                       server = "tcp:<server>database.windows.net,1433",
                       database = "<db_name>",
                       uid = keyring::key_list("<db_name>")[["username"]],
                       pwd = keyring::key_get("<db_name>", keyring::key_list("<db_name>")[["username"]]),
                       Encrypt = "yes",
                       TrustServerCertificate = "yes",
                       Authentication = "ActiveDirectoryPassword")

DBI::dbExecute(con, "COPY INTO <schema>.<table>
(<field_names>)
FROM 'https://<blob_storage_path>.gz'
WITH (
  FILE_TYPE = 'csv',
  CREDENTIAL = (IDENTITY = 'Storage Account Key',
SECRET = '<key_that_contains_alphanumeric_characters_plus_/_and_+_and_padding=='),
  MAXERRORS = 100,
  COMPRESSION = 'gzip',
  FIELDQUOTE = ''  ,
  FIELDTERMINATOR = '\t',
  ROWTERMINATOR = '\n',
  FIRSTROW = 2
);")
hadley commented 1 year ago

Possibly related to #367