search

tomroh / bcputility

R package for fast bulk imports/exports from/to SQL Server with the bcp command line utility
https://bcputility.roh.engineering
Other
13 stars 1 forks source link

Synapse compatibility #22

Closed jtisack closed 3 months ago

jtisack commented 4 months ago

This PR includes two changes we (@sclewis23) found necessary to make the package compatible with our environment (Linux client and Azure Synapse database).

  1. Synapse will only interface with bcp and sqlcmd if QUOTED_IDENTIFIERS is set to ON via CLI flags. Currently bcputility allows passing additional arguments to bcp CLI but not sqlcmd. This change adds a single argument to makeConnectArgs() that adds the flag to all calls to both bcp and sqlcmd.

  2. We found that adding brackets around table names ("quoting") in the bcp CLI causes an issue when importing data. This change adds an argument to bcpImport() that disables that functionality.

Documentation on the added bcp/sqlcmd arguments is here: https://learn.microsoft.com/en-us/sql/tools/bcp-utility?view=sql-server-ver16#-q https://learn.microsoft.com/en-us/sql/tools/sqlcmd/sqlcmd-utility?view=sql-server-ver16&tabs=go%2Clinux&pivots=cs1-bash#-i

netlify[bot] commented 4 months ago

Deploy Preview for bcputility canceled.

Name Link
Latest commit d862c696422b1a88b728a5747fb32076c6e8227a
Latest deploy log https://app.netlify.com/sites/bcputility/deploys/65f213b88ee07500090623ff
tomroh commented 3 months ago

@jtisack just want to let you know this is on my radar. I just need to find time to get a test environment set up.

tomroh commented 3 months ago

@jtisack, should all be sorted out now. You can try the devel branch with the changes and see if it it working for your config.

devtools::install_github("tomroh/bcputility@devel")
jtisack commented 3 months ago

@tomroh We tested the devel branch internally and it's all working perfectly. Thanks!

tomroh commented 3 months ago

@jtisack Great! Thanks for the help! Will be on CRAN in the next few weeks.