snowflakedb / dplyr-snowflakedb

SnowflakeDB backend for dplyr
Apache License 2.0
65 stars 25 forks source link

Is dplyr.snowflakedb only running on 32-bit? #30

Open quotechltd opened 4 years ago

quotechltd commented 4 years ago

I cannot install the package on my Windows 10 64-bit PC. I've got the latest version of R installed which was required to install RJDBC and rJava is loading and running without any issue on all the other packages I use with a dependency to it. Here are the results of running sessionInfo():

R version 3.6.2 (2019-12-12) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 18363)

Matrix products: default

Random number generation: RNG: Mersenne-Twister Normal: Inversion Sample: Rounding

locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252
[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C
[5] LC_TIME=English_United Kingdom.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

loaded via a namespace (and not attached): [1] compiler_3.6.2 tools_3.6.2 mongolite_2.1.0 jsonlite_1.6.1

Here's the result of running Sys.getenv("JAVA_HOME"):

"C:\Program Files\Java\jdk-13.0.2"

Here's the result of running library(rJava) .jinit() .jcall("java/lang/System", "S", "getProperty", "java.runtime.version")

"13.0.2+8"

However, here's the result of running devtools::install_github("snowflakedb/dplyr-snowflakedb"):

[...] *** arch - i386 Error: package or namespace load failed for 'rJava': .onLoad failed in loadNamespace() for 'rJava', details: call: inDL(x, as.logical(local), as.logical(now), ...) error: unable to load shared object 'C:/Users/[...]/R/win-library/3.6/rJava/libs/i386/rJava.dll': LoadLibrary failure: %1 is not a valid Win32 application. [...]

Is the package only working if you have a 32-bit version of JDK installed?

mark-druffel commented 4 years ago

@quotechltd did you ever figure out a solution? I'm having the same issue. I've tried installing using R 32-bit, changing my JAVA_HOME path, and a few other suggestions I found on Stackoverflow to no avail.

quotechltd commented 4 years ago

@mark-druffel unfortunately I didn't.

mark-druffel commented 4 years ago

@quotechltd I was able to use the python connector through reticulate, but it comes with some baggage because you get back lists with python data types that are painful to coerce into a data frame. Below is how I was able to get it working, but the snowflake_to_df function cannot handle those python lists I mentioned (e.g. date columns). I'm still trying to figure out how to create a generic function to solve that.

` library(reticulate) py_install('snowflake-connector-python') py_install('snowflake-connector-python[pandas]') py_install('snowflake-sqlalchemy') snowflake.connector <- import('snowflake.connector') con = snowflake.connector$connect( user=, account=, authenticator='externalbrowser', warehouse=, database=, schema=, role=)

tables_columns <- con$cursor()$execute('Show Columns')$fetchall() col_names <- con$cursor()$execute('Show Columns')$description col_names <- purrr::map(col_names, function(x) purrr::pluck(x, 1)) %>% unlist() snowflake_to_df <- function(data=NULL, column_names=NULL,keep_columns=NULL){ if(is.null(keep_columns)){keep_columns <- rep_len(T, length(column_names))} x <- purrr::set_names(data, column_names) %>% purrr::keep(., keep_columns) return(x) } tables_columns <- purrr::map(tables_columns, snowflake_to_df, column_names = col_names) %>% dplyr::bind_rows() `

quotechltd commented 4 years ago

Thanks @mark-druffel I will try that when I next try to connect to Snowflake. I had put that on hold for a while.