search

sparklyr / sparklyr

R interface for Apache Spark
https://spark.rstudio.com/
Apache License 2.0
948 stars 307 forks source link

R session crashes after trying to use spark_connect for the first time #1085

Open mwolfe07 opened 6 years ago

mwolfe07 commented 6 years ago

Reporting an Issue with sparklyr

I'm having issues trying to connect using sparklyr - have just installed it. 

install.packages('sparklyr')
require(sparklyr)
spark_install()
sc <- spark_connect(master = "local")

Ive had a few errors I worked through like my dplyr version not being up to date, and something about not having java on my machine. So I downloaded the most recent version of java, and updated my packages. I'm receiving the following error still:

Using Spark: 2.1.0 Fatal error: you must specify '--save', '--no-save' or '--vanilla'
And my R session crashes. Any help would be greatly appreciated! Thanks
your brief description of the problem

I've tried calling a different version of Spark when running spark_connect, but with no luck. I'm also not sure how to create a reproducible example of this error, if anyone has any tips, it would be greatly appreciated!

javierluraschi commented 6 years ago

@mwolfe07 could you please share your sessionInfo()? Thanks.

mwolfe07 commented 6 years ago

@javierluraschi

I've updated the sessionInfo() below. Thanks!

R version 3.4.1 (2017-06-30)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252    LC_MONETARY=English_United States.1252 LC_NUMERIC=C                          
[5] LC_TIME=English_United States.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] readr_1.1.1         sparklyr_0.6.3      pryr_0.1.2          lubridate_1.6.0     stringr_1.2.0       RForcecom_1.1       gdata_2.18.0       
 [8] purrr_0.2.2.2       tidyr_0.6.3         dplyr_0.7.4         data.table_1.10.4   scales_0.4.1        ggplot2_2.2.1       LogRIn2_0.0.0.9000 
[15] devtools_1.13.2     roxygen2_6.0.1.9000

loaded via a namespace (and not attached):
 [1] gtools_3.5.0     reshape2_1.4.2   lattice_0.20-35  colorspace_1.3-2 htmltools_0.3.6  base64enc_0.1-3  XML_3.98-1.9     rlang_0.1.2      foreign_0.8-69  
[10] glue_1.1.1       withr_2.0.0      DBI_0.7          dbplyr_1.1.0     bindrcpp_0.2     bindr_0.1        plyr_1.8.4       munsell_0.4.3    commonmark_1.4  
[19] gtable_0.2.0     psych_1.7.8      codetools_0.2-15 memoise_1.1.0    httpuv_1.3.5     parallel_3.4.1   broom_0.4.2      Rcpp_0.12.13     xtable_1.8-2    
[28] backports_1.1.1  desc_1.1.1       jsonlite_1.5     mime_0.5         mnormt_1.5-5     hms_0.3          digest_0.6.12    stringi_1.1.5    shiny_1.0.5     
[37] grid_3.4.1       rprojroot_1.2    tools_3.4.1      bitops_1.0-6     magrittr_1.5     lazyeval_0.2.0   RCurl_1.95-4.8   tibble_1.3.4     crayon_1.3.4    
[46] pkgconfig_2.0.1  xml2_1.1.1       assertthat_0.2.0 httr_1.3.1       rstudioapi_0.7   R6_2.2.2         nlme_3.1-131     compiler_3.4.1