search

greta-dev / greta

simple and scalable statistical modelling in R
https://greta-stats.org
Other
524 stars 63 forks source link

greta has crash situation: address 0x50, cause 'memory not mapped' #565

Closed BillyChen123 closed 1 year ago

BillyChen123 commented 1 year ago

address 0x50, cause 'memory not mapped'

I followed the standard process of installing greta on the server. But when I started to use it, my R session was accidently crashed. The situation was similar with the problem #526 but has the different error. I am pretty sure that my tensorflow was installed perfectly. But when I run the basic function in greta like normal(0,1). The R session was crashed. I took reprex(greta::greta_sitrep(), std_out_err = TRUE, si = TRUE) to examine the cause of error. the result was as follow:

reprex(greta::greta_sitrep(), std_out_err = TRUE, si = TRUE)

This reprex appears to crash R. See standard output and standard error for more details.

Standard output and error


 *** caught segfault ***
address 0x50, cause 'memory not mapped'

Traceback:
 1: py_module_import(module, convert = convert)
 2: import(module)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch({    import(module)    TRUE}, error = clear_error_handler(FALSE))
 7: reticulate::py_module_available("tensorflow")
 8: have_tf()
 9: check_if_software_available(software_available = have_tf(), version = version_tf(),     ideal_version = "1.14.0", software_name = "TensorFlow")
10: greta::greta_sitrep()
11: eval(expr, envir, enclos)
12: eval(expr, envir, enclos)
13: eval_with_user_handlers(expr, envir, enclos, user_handlers)
14: withVisible(eval_with_user_handlers(expr, envir, enclos, user_handlers))
15: withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)
16: doTryCatch(return(expr), name, parentenv, handler)
17: tryCatchOne(expr, names, parentenv, handlers[[1L]])
18: tryCatchList(expr, classes, parentenv, handlers)
19: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
20: try(f, silent = TRUE)
21: handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler))
22: timing_fn(handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)))
23: evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos,     debug = debug, last = i == length(out), use_try = stop_on_error !=         2L, keep_warning = keep_warning, keep_message = keep_message,     output_handler = output_handler, include_timing = include_timing)
24: evaluate::evaluate(...)
25: evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning),     keep_message = !isFALSE(options$message), stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options))
26: in_dir(input_dir(), expr)
27: in_input_dir(evaluate(code, envir = env, new_device = FALSE,     keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message),     stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options)))
28: eng_r(options)
29: block_exec(params)
30: call_block(x)
31: process_group.block(group)
32: process_group(group)
33: withCallingHandlers(if (tangle) process_tangle(group) else process_group(group),     error = function(e) {        setwd(wd)        cat(res, sep = "\n", file = output %n% "")        message("Quitting from lines ", paste(current_lines(i),             collapse = "-"), " (", knit_concord$get("infile"),             ") ")    })
34: process_file(text, output)
35: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
36: rmarkdown::render(input, quiet = TRUE, envir = globalenv(), encoding = "UTF-8")
37: (function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")})(input = base::quote("next-esok_reprex.R"))
38: (function (what, args, quote = FALSE, envir = parent.frame()) {    if (!is.list(args))         stop("second argument must be a list")    if (quote)         args <- lapply(args, enquote)    .Internal(do.call(what, args, envir))})(base::quote(function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")}), base::quote(list(input = "next-esok_reprex.R")), envir = base::quote(<environment>),     quote = base::quote(TRUE))
39: do.call(do.call, c(readRDS("/tmp/RtmpeArnA9/callr-fun-96445df5949"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE)
40: saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpeArnA9/callr-fun-96445df5949"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE), file = "/tmp/RtmpeArnA9/callr-res-96429df7bd9",     compress = FALSE)
41: withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpeArnA9/callr-fun-96445df5949"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpeArnA9/callr-res-96429df7bd9",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpeArnA9/callr-res-96429df7bd9",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpeArnA9/callr-res-96429df7bd9",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))})
42: doTryCatch(return(expr), name, parentenv, handler)
43: tryCatchOne(expr, names, parentenv, handlers[[1L]])
44: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
45: doTryCatch(return(expr), name, parentenv, handler)
46: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]),     names[nh], parentenv, handlers[[nh]])
47: tryCatchList(expr, classes, parentenv, handlers)
48: tryCatch(withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpeArnA9/callr-fun-96445df5949"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpeArnA9/callr-res-96429df7bd9",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpeArnA9/callr-res-96429df7bd9",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpeArnA9/callr-res-96429df7bd9",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))}), error = function(e) {    NULL    try(stop(e))}, interrupt = function(e) {    NULL    e})
An irrecoverable exception occurred. R is aborting now ...
njtierney commented 1 year ago

Hi @BillyChen123

Thanks for posting an issue!

Regarding the issue https://github.com/greta-dev/greta/issues/526 , the error ended up being that the version of R installed was for intel macs, but the user needed to use the arm64 (M1 chip version) installation.

Can you post the output of the session info from the reprex? I believe this should have been attached as you had the option si = TRUE in your reprex, but you can also return the session info information with:

reprex::reprex(x = 1, session_info = TRUE)

And then paste that output in? Just so I can see the details of your R session and R installation.

Hopefully we can resolve this!

In the interim, perhaps see if the solution in https://github.com/rstudio/tensorflow/issues/537 may help resolve this for you?

BillyChen123 commented 1 year ago

Thank you for your advice. I have just run the recommended code you mention above, and it seems didn't have error.

1
#> [1] 1

Created on 2022-10-13 with reprex v2.0.2

Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-13 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

I have already seen the solution of #526. I also run the code as follow:

''' reprex::reprex({ library(tensorflow) as_tensor(1) }, std_out_err = TRUE, si = TRUE ) '''

Here is the results:

library(tensorflow)
as_tensor(1)
#> Loaded Tensorflow version 2.9.2
#> tf.Tensor(1.0, shape=(), dtype=float32)

Created on 2022-10-13 with reprex v2.0.2

Standard output and standard error ``` sh 2022-10-13 13:43:35.314847: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`. 2022-10-13 13:43:35.916666: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/software/R-4.0.2/lib64/R/lib:/usr/local/lib64:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-1.el7.x86_64/jre/lib/amd64/server:/opt/software/R-4.0.2/lib64/R/lib::/lib:/usr/local/lib64:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-1.el7.x86_64/jre/lib/amd64/server 2022-10-13 13:43:35.916708: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine. 2022-10-13 13:44:10.906048: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/software/R-4.0.2/lib64/R/lib:/usr/local/lib64:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-1.el7.x86_64/jre/lib/amd64/server:/opt/software/R-4.0.2/lib64/R/lib::/lib:/usr/local/lib64:/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.262.b10-1.el7.x86_64/jre/lib/amd64/server 2022-10-13 13:44:10.906100: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303) 2022-10-13 13:44:10.906130: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (server): /proc/driver/nvidia/version does not exist 2022-10-13 13:44:10.925345: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 AVX512F AVX512_VNNI FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. ```
Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-13 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> base64enc 0.1-3 2015-07-28 [2] CRAN (R 4.0.2) #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> here 1.0.1 2020-12-13 [2] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> jsonlite 1.8.2 2022-10-02 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> Matrix 1.5-1 2022-09-13 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> png 0.1-7 2013-12-03 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> rappdirs 0.3.3 2021-01-31 [2] CRAN (R 4.0.2) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> reticulate 1.26 2022-08-31 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rprojroot 2.0.3 2022-04-02 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tensorflow * 2.9.0 2022-05-21 [1] CRAN (R 4.0.2) #> tfruns 1.5.1 2022-09-05 [1] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> whisker 0.4 2019-08-28 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ─ Python configuration ─────────────────────────────────────────────────────── #> python: /home/chenyz/.local/share/r-miniconda/envs/r-reticulate/bin/python #> libpython: /home/chenyz/.local/share/r-miniconda/envs/r-reticulate/lib/libpython3.8.so #> pythonhome: /home/chenyz/.local/share/r-miniconda/envs/r-reticulate:/home/chenyz/.local/share/r-miniconda/envs/r-reticulate #> version: 3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:04:18) [GCC 10.3.0] #> numpy: /home/chenyz/.local/share/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/numpy #> numpy_version: 1.23.3 #> #> ────────────────────────────────────────────────────────────────────────────── ```

I think that is not the problem of tensorflow.

njtierney commented 1 year ago

Hi @BillyChen123 - thanks for this.

It looks like you are on Linux - which is good, I thought that the error might be tied up to using an M1 mac, which is seems it isn't!

The other reprex with 1 in it was just to get out the session info that you have also provided in the next example, so that is all good :)

Can you try:

reprex::reprex(library(greta),
  std_out_err = TRUE,
  si = TRUE
)

And paste the information in?

The next thing I would recommend is for you to use the greta::reinstall_greta_deps() function to do a fresh installation of the python dependencies.

BillyChen123 commented 1 year ago

Thank you! I follow your advice, and there is still not error.

library(greta)
#> 
#> Attaching package: 'greta'
#> The following objects are masked from 'package:stats':
#> 
#>     binomial, cov2cor, poisson
#> The following objects are masked from 'package:base':
#> 
#>     %*%, apply, backsolve, beta, chol2inv, colMeans, colSums, diag,
#>     eigen, forwardsolve, gamma, identity, rowMeans, rowSums, sweep,
#>     tapply

Created on 2022-10-13 with reprex v2.0.2

Standard output and standard error ``` sh -- nothing to show -- ```
Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-13 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> base64enc 0.1-3 2015-07-28 [2] CRAN (R 4.0.2) #> callr 3.7.2 2022-08-22 [2] CRAN (R 4.0.2) #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> coda 0.19-4 2020-09-30 [2] CRAN (R 4.0.2) #> codetools 0.2-18 2020-11-04 [2] CRAN (R 4.0.2) #> crayon 1.5.2 2022-09-29 [2] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> ellipsis 0.3.2 2021-04-29 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> future 1.28.0 2022-09-02 [2] CRAN (R 4.0.2) #> globals 0.16.1 2022-08-28 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> greta * 0.4.3 2022-09-08 [1] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> hms 1.1.2 2022-08-19 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> jsonlite 1.8.2 2022-10-02 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> listenv 0.8.0 2019-12-05 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> Matrix 1.5-1 2022-09-13 [2] CRAN (R 4.0.2) #> parallelly 1.32.1 2022-07-21 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> png 0.1-7 2013-12-03 [2] CRAN (R 4.0.2) #> prettyunits 1.1.1 2020-01-24 [2] CRAN (R 4.0.2) #> processx 3.7.0 2022-07-07 [2] CRAN (R 4.0.2) #> progress 1.2.2 2019-05-16 [2] CRAN (R 4.0.2) #> ps 1.7.1 2022-06-18 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> R6 2.5.1 2021-08-19 [2] CRAN (R 4.0.2) #> rappdirs 0.3.3 2021-01-31 [2] CRAN (R 4.0.2) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> reticulate 1.26 2022-08-31 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tensorflow 2.9.0 2022-05-21 [1] CRAN (R 4.0.2) #> tfruns 1.5.1 2022-09-05 [1] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> whisker 0.4 2019-08-28 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```

Then I use the ''' greta::reinstall_greta_deps() ''' function to fresh installation of the python dependencies.

There is a decision that I have to make:

Are you sure you want to delete miniconda from /home/chenyz/.local/share/r-miniconda? 1: No 2: Absolutely 3: Nope

Could you tell me which one should I choose?

BillyChen123 commented 1 year ago

Hello, sorry to disturb you. I have already use the '''greta::reinstall_greta_deps()''' function. But it still can't work. The error remains: address 0x50, cause 'memory not mapped' What should I do next?

njtierney commented 1 year ago

Hi @BillyChen123 !

RE,

Are you sure you want to delete miniconda from /home/chenyz/.local/share/r-miniconda? 1: No 2: Absolutely 3: Nope

Could you tell me which one should I choose?

You should select the option that aligns with "yes" - this option and name of the option will change each time, which is by design - sometimes it is "yes, no, maybe", and other times something else entirely. But you want to remove it.

njtierney commented 1 year ago

The next thing you can do is, just as you have made that lovely reprex above with no error, if you can do the same but for the code that errors, so I can get more information, hopefully we can work this out!

BillyChen123 commented 1 year ago

@njtierney Thanks! I follow your advice and rebuild the miniconda environment. Then I run the same reprex again. The error turn to be like this:

greta::greta_sitrep()
#> ℹ checking if python available
#> Warning in py_initialize(config$python, config$libpython, config$pythonhome, :
#> Python 2 reached EOL on January 1, 2020. Python 2 compatability be removed in an
#> upcoming reticulate release.
#> ✖ python available, however 3.7 is needed and 2.7 was detected
#> 
#> ℹ checking if TensorFlow available
#> ✖ TensorFlow not available
#> 
#> ℹ checking if TensorFlow Probability available
#> ✖ TensorFlow Probability not available
#> 
#> ℹ checking if greta conda environment available
#> ✖ greta conda environment not available
#> 
#> ℹ Initialising python and checking dependencies, this may take a moment.
#> ✖ Initialising python and checking dependencies, this may take a moment. ... fa…
#> 
#> Warning: We have detected that you do not have the expected python packages setup.
#> You can set these up by running this R code in the console:
#> `install_greta_deps()`
#> Then, restart R and run:
#> `library(greta)`
#> (Note: Your R session should not have initialised Tensorflow yet.)
#> For more information, see `?install_greta_deps`

Created on 2022-10-17 with reprex v2.0.2

Standard output and standard error ``` sh -- nothing to show -- ```
Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-17 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> base64enc 0.1-3 2015-07-28 [2] CRAN (R 4.0.2) #> callr 3.7.2 2022-08-22 [2] CRAN (R 4.0.2) #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> coda 0.19-4 2020-09-30 [2] CRAN (R 4.0.2) #> codetools 0.2-18 2020-11-04 [2] CRAN (R 4.0.2) #> crayon 1.5.2 2022-09-29 [2] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> ellipsis 0.3.2 2021-04-29 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> future 1.28.0 2022-09-02 [2] CRAN (R 4.0.2) #> globals 0.16.1 2022-08-28 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> greta 0.4.3 2022-09-08 [1] CRAN (R 4.0.2) #> here 1.0.1 2020-12-13 [2] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> hms 1.1.2 2022-08-19 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> jsonlite 1.8.2 2022-10-02 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> listenv 0.8.0 2019-12-05 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> Matrix 1.5-1 2022-09-13 [2] CRAN (R 4.0.2) #> parallelly 1.32.1 2022-07-21 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> png 0.1-7 2013-12-03 [2] CRAN (R 4.0.2) #> prettyunits 1.1.1 2020-01-24 [2] CRAN (R 4.0.2) #> processx 3.7.0 2022-07-07 [2] CRAN (R 4.0.2) #> progress 1.2.2 2019-05-16 [2] CRAN (R 4.0.2) #> ps 1.7.1 2022-06-18 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> R6 2.5.1 2021-08-19 [2] CRAN (R 4.0.2) #> rappdirs 0.3.3 2021-01-31 [2] CRAN (R 4.0.2) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> reticulate 1.26 2022-08-31 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rprojroot 2.0.3 2022-04-02 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tensorflow 2.9.0 2022-05-21 [1] CRAN (R 4.0.2) #> tfruns 1.5.1 2022-09-05 [1] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> whisker 0.4 2019-08-28 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ─ Python configuration ─────────────────────────────────────────────────────── #> python: /usr/bin/python #> libpython: /usr/lib64/python2.7/config/libpython2.7.so #> pythonhome: //usr://usr #> version: 2.7.5 (default, Oct 14 2020, 14:45:30) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)] #> numpy: /usr/lib64/python2.7/site-packages/numpy #> numpy_version: 1.7.1 #> tensorflow: [NOT FOUND] #> #> ────────────────────────────────────────────────────────────────────────────── ```

So, should I run 'install_greta_deps()' again?

njtierney commented 1 year ago

Hi @BillyChen123

Thanks!

I would suggest running greta::reinstall_greta_deps() and if that doesn't work, greta::install_greta_deps().

Let me know how you go?

BillyChen123 commented 1 year ago

Hi!@njtierney Oh no! There is another accident happened when I run greta::reinstall_greta_deps()

Selection: 2 ℹ removing 'miniconda' installation ✔ 'miniconda' successfully removed! ✖ No miniconda detected, installing miniconda, this may take a minute. ... failed

njtierney commented 1 year ago

Hi @BillyChen123 - looks like you might need more time to install, it also looks like an error message got chopped off in your issue post above?

I'd recommend increasing the timeout for installation, using the code provided below.

install_greta_deps(timeout = 15)

Alternatively if that doesn't work, you could try following the alternative installation instructions that are provided in the message in your post above?

Alternatively, you can perform the entire installation with: reticulate::install_miniconda() Then: reticulate::conda_create(envname = 'greta-env', python_version = '3.7') Then: reticulate::conda_install(envname = 'greta-env', packages = c('numpy==1.16.4', 'tensorflow-probability==0.7.0', 'tensorflow==1.14.0')) Then, restart R, and load greta with: library(greta)

BillyChen123 commented 1 year ago

Hi! @njtierney I run the second suggest you have mention. It seems that my tensorflow probobility can not install. What should I do next?

reticulate::conda_install(envname = 'greta-env', packages = c('numpy==1.16.4', 'tensorflow-probability==0.7.0', 'tensorflow==1.14.0'))

Package Plan

environment location: /home/chenyz/.local/share/r-miniconda/envs/greta-env

added / updated specs:

The following packages will be downloaded:

package                    |            build
---------------------------|-----------------
absl-py-1.3.0              |     pyhd8ed1ab_0          95 KB  conda-forge
decorator-5.1.1            |     pyhd8ed1ab_0          12 KB  conda-forge
gast-0.5.3                 |     pyhd8ed1ab_0          20 KB  conda-forge
grpc-cpp-1.48.1            |       h30feacc_1         5.4 MB  conda-forge
grpcio-1.48.1              |   py37he7b19e7_1         819 KB  conda-forge
importlib-metadata-4.11.4  |   py37h89c1867_0          33 KB  conda-forge
keras-preprocessing-1.1.2  |     pyhd8ed1ab_0          34 KB  conda-forge
krb5-1.19.3                |       h08a2579_0         1.4 MB  conda-forge
libcblas-3.9.0             |       8_openblas          11 KB  conda-forge
libcurl-7.85.0             |       h2283fc2_0         351 KB  conda-forge
libev-4.33                 |       h516909a_1         104 KB  conda-forge
libgfortran-ng-7.5.0       |      h14aa051_20          23 KB  conda-forge
liblapack-3.9.0            |       8_openblas          11 KB  conda-forge
libprotobuf-3.21.7         |       h6239696_0         2.5 MB  conda-forge
libssh2-1.10.0             |       hf14f497_3         234 KB  conda-forge
markupsafe-2.1.1           |   py37h540881e_1          22 KB  conda-forge
numpy-1.16.4               |   py37h95a1406_0         4.3 MB  conda-forge
python_abi-3.7             |          2_cp37m           4 KB  conda-forge
re2-2022.06.01             |       h27087fc_0         215 KB  conda-forge
six-1.16.0                 |     pyh6c4a22f_0          14 KB  conda-forge
tensorflow-base-1.14.0     |   py37h4531e10_0        87.6 MB  conda-forge
tensorflow-estimator-1.14.0|   py37h5ca1d4c_0         645 KB  conda-forge
termcolor-2.0.1            |     pyhd8ed1ab_1           9 KB  conda-forge
typing_extensions-4.4.0    |     pyha770c72_0          29 KB  conda-forge
werkzeug-2.2.2             |     pyhd8ed1ab_0         249 KB  conda-forge
zipp-3.9.0                 |     pyhd8ed1ab_0          13 KB  conda-forge
------------------------------------------------------------
                                       Total:       104.1 MB

The following NEW packages will be INSTALLED:

absl-py conda-forge/noarch::absl-py-1.3.0-pyhd8ed1ab_0 None astor conda-forge/noarch::astor-0.8.1-pyh9f0ad1d_0 None c-ares conda-forge/linux-64::c-ares-1.18.1-h7f98852_0 None cloudpickle conda-forge/noarch::cloudpickle-2.2.0-pyhd8ed1ab_0 None decorator conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0 None gast conda-forge/noarch::gast-0.5.3-pyhd8ed1ab_0 None google-pasta conda-forge/noarch::google-pasta-0.2.0-pyh8c360ce_0 None grpc-cpp conda-forge/linux-64::grpc-cpp-1.48.1-h30feacc_1 None grpcio conda-forge/linux-64::grpcio-1.48.1-py37he7b19e7_1 None h5py conda-forge/linux-64::h5py-2.10.0-nompi_py37h90cd8ad_104 None hdf5 conda-forge/linux-64::hdf5-1.10.6-nompi_h3c11f04_101 None importlib-metadata conda-forge/linux-64::importlib-metadata-4.11.4-py37h89c1867_0 None keras-applications conda-forge/noarch::keras-applications-1.0.8-py_1 None keras-preprocessi~ conda-forge/noarch::keras-preprocessing-1.1.2-pyhd8ed1ab_0 None keyutils conda-forge/linux-64::keyutils-1.6.1-h166bdaf_0 None krb5 conda-forge/linux-64::krb5-1.19.3-h08a2579_0 None libabseil conda-forge/linux-64::libabseil-20220623.0-cxx17_h48a1fff_4 None libblas conda-forge/linux-64::libblas-3.9.0-8_openblas None libcblas conda-forge/linux-64::libcblas-3.9.0-8_openblas None libcurl conda-forge/linux-64::libcurl-7.85.0-h2283fc2_0 None libedit conda-forge/linux-64::libedit-3.1.20191231-he28a2e2_2 None libev conda-forge/linux-64::libev-4.33-h516909a_1 None libgfortran-ng conda-forge/linux-64::libgfortran-ng-7.5.0-h14aa051_20 None libgfortran4 conda-forge/linux-64::libgfortran4-7.5.0-h14aa051_20 None liblapack conda-forge/linux-64::liblapack-3.9.0-8_openblas None libnghttp2 conda-forge/linux-64::libnghttp2-1.47.0-hff17c54_1 None libopenblas conda-forge/linux-64::libopenblas-0.3.12-pthreads_hb3c22a3_1 None libpng conda-forge/linux-64::libpng-1.6.38-h753d276_0 None libprotobuf conda-forge/linux-64::libprotobuf-3.21.7-h6239696_0 None libssh2 conda-forge/linux-64::libssh2-1.10.0-hf14f497_3 None markdown conda-forge/noarch::markdown-3.4.1-pyhd8ed1ab_0 None markupsafe conda-forge/linux-64::markupsafe-2.1.1-py37h540881e_1 None numpy conda-forge/linux-64::numpy-1.16.4-py37h95a1406_0 None protobuf conda-forge/linux-64::protobuf-4.21.7-py37hd23a5d3_0 None python_abi conda-forge/linux-64::python_abi-3.7-2_cp37m None re2 conda-forge/linux-64::re2-2022.06.01-h27087fc_0 None scipy conda-forge/linux-64::scipy-1.5.1-py37ha3d9a3c_0 None six conda-forge/noarch::six-1.16.0-pyh6c4a22f_0 None tensorboard conda-forge/linux-64::tensorboard-1.14.0-py37_0 None tensorflow conda-forge/linux-64::tensorflow-1.14.0-h4531e10_0 None tensorflow-base conda-forge/linux-64::tensorflow-base-1.14.0-py37h4531e10_0 None tensorflow-estima~ conda-forge/linux-64::tensorflow-estimator-1.14.0-py37h5ca1d4c_0 None tensorflow-probab~ conda-forge/noarch::tensorflow-probability-0.7-py_3 None termcolor conda-forge/noarch::termcolor-2.0.1-pyhd8ed1ab_1 None typing_extensions conda-forge/noarch::typing_extensions-4.4.0-pyha770c72_0 None werkzeug conda-forge/noarch::werkzeug-2.2.2-pyhd8ed1ab_0 None wrapt conda-forge/linux-64::wrapt-1.14.1-py37h540881e_0 None zipp conda-forge/noarch::zipp-3.9.0-pyhd8ed1ab_0 None zlib conda-forge/linux-64::zlib-1.2.13-h166bdaf_4 None

Downloading and Extracting Packages libcurl-7.85.0 | 351 KB | ########## | 100% tensorflow-base-1.14 | 87.6 MB | #########7 | 97% libssh2-1.10.0 | 234 KB | ########## | 100% liblapack-3.9.0 | 11 KB | ########## | 100% keras-preprocessing- | 34 KB | ########## | 100% libprotobuf-3.21.7 | 2.5 MB | ########## | 100% libcblas-3.9.0 | 11 KB | ########## | 100% decorator-5.1.1 | 12 KB | ########## | 100% six-1.16.0 | 14 KB | ########## | 100% libev-4.33 | 104 KB | ########## | 100% importlib-metadata-4 | 33 KB | ########## | 100% tensorflow-estimator | 645 KB | ########## | 100% grpcio-1.48.1 | 819 KB | ########## | 100% numpy-1.16.4 | 4.3 MB | ########## | 100% krb5-1.19.3 | 1.4 MB | ########## | 100% python_abi-3.7 | 4 KB | ########## | 100% markupsafe-2.1.1 | 22 KB | ########## | 100% absl-py-1.3.0 | 95 KB | ########## | 100% termcolor-2.0.1 | 9 KB | ########## | 100% libgfortran-ng-7.5.0 | 23 KB | ########## | 100% grpc-cpp-1.48.1 | 5.4 MB | ########## | 100% gast-0.5.3 | 20 KB | ########## | 100% zipp-3.9.0 | 13 KB | ########## | 100% typing_extensions-4. | 29 KB | ########## | 100% werkzeug-2.2.2 | 249 KB | ########## | 100% re2-2022.06.01 | 215 KB | ########## | 100% CondaError: Downloaded bytes did not match Content-Length url: https://conda.anaconda.org/conda-forge/linux-64/tensorflow-base-1.14.0-py37h4531e10_0.tar.bz2 target_path: /home/chenyz/.local/share/r-miniconda/pkgs/tensorflow-base-1.14.0-py37h4531e10_0.tar.bz2 Content-Length: 91901368 downloaded bytes: 89369019

Error: one or more Python packages failed to install [error code 1]

njtierney commented 1 year ago

Hi @BillyChen123

So just to confirm, you ran the following code

reticulate::install_miniconda()

Then you ran:

reticulate::conda_create(envname = 'greta-env', python_version = '3.7')

Then you ran

reticulate::conda_install(envname = 'greta-env', packages = c('numpy==1.16.4', 'tensorflow-probability==0.7.0', 'tensorflow==1.14.0'))

Is that correct? Just want to double check - thanks for your patience with this, installation is a tricky business, hopefully we can solve this soon!

BillyChen123 commented 1 year ago

Hi! @njtierney I followed the suggestion from beginning to the end once again. The error remains, but in different way. This time, when I run the code reticulate::install_miniconda(). There has a new error.

reticulate::install_miniconda()

Error in download.file(url, destfile = installer, mode = "wb") : download from 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh' failed In addition: Warning messages: 1: In download.file(url, destfile = installer, mode = "wb") : downloaded length 11745673 != reported length 76607678 2: In download.file(url, destfile = installer, mode = "wb") : URL 'https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh': status was 'Failure when receiving data from the peer'

njtierney commented 1 year ago

Hi @BillyChen123 - thanks for that!

It looks like there might be an issue with installing miniconda.

Can you try the following?

greta::remove_miniconda()

select the appropriate option, then:

reticulate::install_miniconda()

Then, just to see if this helps, try:

reticulate::conda_update()

Then do:

greta::install_greta_deps()

And if the greta install command above doesn't work, then restart R

Then try the following:

reticulate::conda_create(envname = 'greta-env', python_version = '3.7')

Then:

reticulate::conda_install(envname = 'greta-env', packages = c('numpy==1.16.4', 'tensorflow-probability==0.7.0', 'tensorflow==1.14.0'))

Thanks again for posting, hopefully this proves helpful.

If this doesn't work, perhaps there is an issue with installing conda on your linux machine? We shall see!

BillyChen123 commented 1 year ago

Hi! @njtierney I had already run all the code you have mention and it all worked. Then, I run the code reprex(greta::greta_sitrep(), std_out_err = TRUE, si = TRUE) The error remains the same. This reprex appears to crash R. See standard output and standard error for more details.

Standard output and error


 *** caught segfault ***
address 0x50, cause 'memory not mapped'

Traceback:
 1: py_module_import(module, convert = convert)
 2: import(module)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch({    import(module)    TRUE}, error = clear_error_handler(FALSE))
 7: reticulate::py_module_available("tensorflow")
 8: have_tf()
 9: check_if_software_available(software_available = have_tf(), version = version_tf(),     ideal_version = "1.14.0", software_name = "TensorFlow")
10: greta::greta_sitrep()
11: eval(expr, envir, enclos)
12: eval(expr, envir, enclos)
13: eval_with_user_handlers(expr, envir, enclos, user_handlers)
14: withVisible(eval_with_user_handlers(expr, envir, enclos, user_handlers))
15: withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)
16: doTryCatch(return(expr), name, parentenv, handler)
17: tryCatchOne(expr, names, parentenv, handlers[[1L]])
18: tryCatchList(expr, classes, parentenv, handlers)
19: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
20: try(f, silent = TRUE)
21: handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler))
22: timing_fn(handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)))
23: evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos,     debug = debug, last = i == length(out), use_try = stop_on_error !=         2L, keep_warning = keep_warning, keep_message = keep_message,     output_handler = output_handler, include_timing = include_timing)
24: evaluate::evaluate(...)
25: evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning),     keep_message = !isFALSE(options$message), stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options))
26: in_dir(input_dir(), expr)
27: in_input_dir(evaluate(code, envir = env, new_device = FALSE,     keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message),     stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options)))
28: eng_r(options)
29: block_exec(params)
30: call_block(x)
31: process_group.block(group)
32: process_group(group)
33: withCallingHandlers(if (tangle) process_tangle(group) else process_group(group),     error = function(e) {        setwd(wd)        cat(res, sep = "\n", file = output %n% "")        message("Quitting from lines ", paste(current_lines(i),             collapse = "-"), " (", knit_concord$get("infile"),             ") ")    })
34: process_file(text, output)
35: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
36: rmarkdown::render(input, quiet = TRUE, envir = globalenv(), encoding = "UTF-8")
37: (function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")})(input = base::quote("bad-grub_reprex.R"))
38: (function (what, args, quote = FALSE, envir = parent.frame()) {    if (!is.list(args))         stop("second argument must be a list")    if (quote)         args <- lapply(args, enquote)    .Internal(do.call(what, args, envir))})(base::quote(function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")}), base::quote(list(input = "bad-grub_reprex.R")), envir = base::quote(<environment>),     quote = base::quote(TRUE))
39: do.call(do.call, c(readRDS("/tmp/RtmpAZafdJ/callr-fun-118277ef7856"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE)
40: saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpAZafdJ/callr-fun-118277ef7856"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE), file = "/tmp/RtmpAZafdJ/callr-res-11824520d30d",     compress = FALSE)
41: withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpAZafdJ/callr-fun-118277ef7856"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpAZafdJ/callr-res-11824520d30d",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpAZafdJ/callr-res-11824520d30d",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpAZafdJ/callr-res-11824520d30d",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))})
42: doTryCatch(return(expr), name, parentenv, handler)
43: tryCatchOne(expr, names, parentenv, handlers[[1L]])
44: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
45: doTryCatch(return(expr), name, parentenv, handler)
46: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]),     names[nh], parentenv, handlers[[nh]])
47: tryCatchList(expr, classes, parentenv, handlers)
48: tryCatch(withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpAZafdJ/callr-fun-118277ef7856"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpAZafdJ/callr-res-11824520d30d",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpAZafdJ/callr-res-11824520d30d",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpAZafdJ/callr-res-11824520d30d",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))}), error = function(e) {    NULL    try(stop(e))}, interrupt = function(e) {    NULL    e})
An irrecoverable exception occurred. R is aborting now ...
njtierney commented 1 year ago

Hi @BillyChen123 !

Thanks for the reprex on this - It looks like the standard output and session info got lopped off in the message above, can you update them or post another example? You can see how it will look on github by clicking the "preview" tab if you want to check that they are there. We're looking for the part that looks like this:

image

Looking further at the error you are getting, I came across this thread on stack overflow:

https://stackoverflow.com/questions/49190251/caught-segfault-memory-not-mapped-error-in-r

I'm wondering if maybe you've updated R recently, and if so, have you had this issue with other libraries, like the tidyverse, or stringi?

BillyChen123 commented 1 year ago

Hi! @njtierney The report is just like this. There is no Session info in the report. Besides, I can run tidyverse and stringi.

njtierney commented 1 year ago

Hi @BillyChen123

The report is just like this. There is no Session info in the report.

Hmmm! So it sounds like the reprex code isn't capturing the session info - normally there should be those little arrows that allow you to see the extra information, but if the reprex you ran didn't provide them, perhaps there is a some issue with the bug interferring with the reprex package.

We could try running the individual parts of greta_sitrep to try and give us a clue for whether there is a specific part that of the checking that causes the segfault. So you could try the following:

reprex::reprex({
have_python()
reticulate::py_version()
have_tf()
version_tf()
have_tfp()
version_tfp()
have_greta_conda_env()
},
si = TRUE,
std_out_err = TRUE)

And if that errors, you could try each line of code in it's own reprex to see if one of them in particular triggers the error. That would be my approach!

BillyChen123 commented 1 year ago

Hi! @njtierney The report is like this!

have_python()
#> Error in have_python(): could not find function "have_python"
reticulate::py_version()
#> NULL
have_tf()
#> Error in have_tf(): could not find function "have_tf"
version_tf()
#> Error in version_tf(): could not find function "version_tf"
have_tfp()
#> Error in have_tfp(): could not find function "have_tfp"
version_tfp()
#> Error in version_tfp(): could not find function "version_tfp"
have_greta_conda_env()
#> Error in have_greta_conda_env(): could not find function "have_greta_conda_env"

Created on 2022-10-19 with reprex v2.0.2

Standard output and standard error ``` sh -- nothing to show -- ```
Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-19 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> jsonlite 1.8.2 2022-10-02 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> Matrix 1.5-1 2022-09-13 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> png 0.1-7 2013-12-03 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> reticulate 1.26 2022-08-31 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```
njtierney commented 1 year ago

Ack, my apologies!

Try:

reprex::reprex({
greta:::have_python()
reticulate::py_version()
greta:::have_tf()
greta:::version_tf()
greta:::have_tfp()
greta:::version_tfp()
greta:::have_greta_conda_env()
},
si = TRUE,
std_out_err = TRUE)

These functions aren't exported, so using ::: allows you to access every function in the greta package - which isn't always recommended but in this case is fine

njtierney commented 1 year ago

And failing this, I would suggest that perhaps you might benefit from upgrading your version of R - which I know can be a pain to reinstall packages again, but that version of R is about 2 years old and it could well be the cause of the issue in some way.

BillyChen123 commented 1 year ago 
reticulate::py_version()
#> NULL

Created on 2022-10-19 with reprex v2.0.2

Standard output and standard error ``` sh -- nothing to show -- ```
Session info ``` r sessioninfo::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os CentOS Linux 7 (Core) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> ctype en_US.UTF-8 #> tz Asia/Shanghai #> date 2022-10-19 #> pandoc 2.7.3 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown) #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> package * version date (UTC) lib source #> cli 3.4.1 2022-09-23 [1] CRAN (R 4.0.2) #> digest 0.6.29 2021-12-01 [2] CRAN (R 4.0.2) #> evaluate 0.17 2022-10-07 [2] CRAN (R 4.0.2) #> fansi 1.0.3 2022-03-24 [2] CRAN (R 4.0.2) #> fastmap 1.1.0 2021-01-25 [2] CRAN (R 4.0.2) #> fs 1.5.2 2021-12-08 [2] CRAN (R 4.0.2) #> glue 1.6.2 2022-02-24 [2] CRAN (R 4.0.2) #> highr 0.9 2021-04-16 [2] CRAN (R 4.0.2) #> htmltools 0.5.3 2022-07-18 [2] CRAN (R 4.0.2) #> jsonlite 1.8.2 2022-10-02 [2] CRAN (R 4.0.2) #> knitr 1.40 2022-08-24 [2] CRAN (R 4.0.2) #> lattice 0.20-45 2021-09-22 [2] CRAN (R 4.0.2) #> lifecycle 1.0.3 2022-10-07 [2] CRAN (R 4.0.2) #> magrittr 2.0.3 2022-03-30 [2] CRAN (R 4.0.2) #> Matrix 1.5-1 2022-09-13 [2] CRAN (R 4.0.2) #> pillar 1.8.1 2022-08-19 [2] CRAN (R 4.0.2) #> pkgconfig 2.0.3 2019-09-22 [2] CRAN (R 4.0.2) #> png 0.1-7 2013-12-03 [2] CRAN (R 4.0.2) #> purrr 0.3.5 2022-10-06 [2] CRAN (R 4.0.2) #> R.cache 0.16.0 2022-07-21 [2] CRAN (R 4.0.2) #> R.methodsS3 1.8.2 2022-06-13 [2] CRAN (R 4.0.2) #> R.oo 1.25.0 2022-06-12 [2] CRAN (R 4.0.2) #> R.utils 2.12.0 2022-06-28 [2] CRAN (R 4.0.2) #> Rcpp 1.0.9 2022-07-08 [1] CRAN (R 4.0.2) #> reprex 2.0.2 2022-08-17 [2] CRAN (R 4.0.2) #> reticulate 1.26 2022-08-31 [2] CRAN (R 4.0.2) #> rlang 1.0.6 2022-09-24 [2] CRAN (R 4.0.2) #> rmarkdown 2.17 2022-10-07 [2] CRAN (R 4.0.2) #> rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.0.2) #> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.0.2) #> stringi 1.7.8 2022-07-11 [2] CRAN (R 4.0.2) #> stringr 1.4.1 2022-08-20 [2] CRAN (R 4.0.2) #> styler 1.7.0 2022-03-13 [2] CRAN (R 4.0.2) #> tibble 3.1.8 2022-07-22 [2] CRAN (R 4.0.2) #> utf8 1.2.2 2021-07-24 [2] CRAN (R 4.0.2) #> vctrs 0.4.2 2022-09-29 [2] CRAN (R 4.0.2) #> withr 2.5.0 2022-03-03 [2] CRAN (R 4.0.2) #> xfun 0.33 2022-09-12 [2] CRAN (R 4.0.2) #> yaml 2.3.5 2022-02-21 [2] CRAN (R 4.0.2) #> #> [1] /home/chenyz/R/x86_64-pc-linux-gnu-library/4.0 #> [2] /opt/software/R-4.0.2/lib64/R/library #> #> ────────────────────────────────────────────────────────────────────────────── ```
BillyChen123 commented 1 year ago

This reprex appears to crash R. See standard output and standard error for more details.

Standard output and error


 *** caught segfault ***
address 0x50, cause 'memory not mapped'

Traceback:
 1: py_module_import(module, convert = convert)
 2: import(module)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch({    import(module)    TRUE}, error = clear_error_handler(FALSE))
 7: reticulate::py_module_available("tensorflow")
 8: greta:::have_tf()
 9: eval(expr, envir, enclos)
10: eval(expr, envir, enclos)
11: eval_with_user_handlers(expr, envir, enclos, user_handlers)
12: withVisible(eval_with_user_handlers(expr, envir, enclos, user_handlers))
13: withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)
14: doTryCatch(return(expr), name, parentenv, handler)
15: tryCatchOne(expr, names, parentenv, handlers[[1L]])
16: tryCatchList(expr, classes, parentenv, handlers)
17: tryCatch(expr, error = function(e) {    call <- conditionCall(e)    if (!is.null(call)) {        if (identical(call[[1L]], quote(doTryCatch)))             call <- sys.call(-4L)        dcall <- deparse(call)[1L]        prefix <- paste("Error in", dcall, ": ")        LONG <- 75L        sm <- strsplit(conditionMessage(e), "\n")[[1L]]        w <- 14L + nchar(dcall, type = "w") + nchar(sm[1L], type = "w")        if (is.na(w))             w <- 14L + nchar(dcall, type = "b") + nchar(sm[1L],                 type = "b")        if (w > LONG)             prefix <- paste0(prefix, "\n  ")    }    else prefix <- "Error : "    msg <- paste0(prefix, conditionMessage(e), "\n")    .Internal(seterrmessage(msg[1L]))    if (!silent && isTRUE(getOption("show.error.messages"))) {        cat(msg, file = outFile)        .Internal(printDeferredWarnings())    }    invisible(structure(msg, class = "try-error", condition = e))})
18: try(f, silent = TRUE)
19: handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler))
20: timing_fn(handle(ev <- withCallingHandlers(withVisible(eval_with_user_handlers(expr,     envir, enclos, user_handlers)), warning = wHandler, error = eHandler,     message = mHandler)))
21: evaluate_call(expr, parsed$src[[i]], envir = envir, enclos = enclos,     debug = debug, last = i == length(out), use_try = stop_on_error !=         2L, keep_warning = keep_warning, keep_message = keep_message,     output_handler = output_handler, include_timing = include_timing)
22: evaluate::evaluate(...)
23: evaluate(code, envir = env, new_device = FALSE, keep_warning = !isFALSE(options$warning),     keep_message = !isFALSE(options$message), stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options))
24: in_dir(input_dir(), expr)
25: in_input_dir(evaluate(code, envir = env, new_device = FALSE,     keep_warning = !isFALSE(options$warning), keep_message = !isFALSE(options$message),     stop_on_error = if (is.numeric(options$error)) options$error else {        if (options$error && options$include)             0L        else 2L    }, output_handler = knit_handlers(options$render, options)))
26: eng_r(options)
27: block_exec(params)
28: call_block(x)
29: process_group.block(group)
30: process_group(group)
31: withCallingHandlers(if (tangle) process_tangle(group) else process_group(group),     error = function(e) {        setwd(wd)        cat(res, sep = "\n", file = output %n% "")        message("Quitting from lines ", paste(current_lines(i),             collapse = "-"), " (", knit_concord$get("infile"),             ") ")    })
32: process_file(text, output)
33: knitr::knit(knit_input, knit_output, envir = envir, quiet = quiet)
34: rmarkdown::render(input, quiet = TRUE, envir = globalenv(), encoding = "UTF-8")
35: (function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")})(input = base::quote("sane-eyas_reprex.R"))
36: (function (what, args, quote = FALSE, envir = parent.frame()) {    if (!is.list(args))         stop("second argument must be a list")    if (quote)         args <- lapply(args, enquote)    .Internal(do.call(what, args, envir))})(base::quote(function (input) {    rmarkdown::render(input, quiet = TRUE, envir = globalenv(),         encoding = "UTF-8")}), base::quote(list(input = "sane-eyas_reprex.R")), envir = base::quote(<environment>),     quote = base::quote(TRUE))
37: do.call(do.call, c(readRDS("/tmp/RtmpKk0QhA/callr-fun-497929e2d840"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE)
38: saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpKk0QhA/callr-fun-497929e2d840"),     list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,     quote = TRUE), file = "/tmp/RtmpKk0QhA/callr-res-49797239d88c",     compress = FALSE)
39: withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpKk0QhA/callr-fun-497929e2d840"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpKk0QhA/callr-res-49797239d88c",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpKk0QhA/callr-res-49797239d88c",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpKk0QhA/callr-res-49797239d88c",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))})
40: doTryCatch(return(expr), name, parentenv, handler)
41: tryCatchOne(expr, names, parentenv, handlers[[1L]])
42: tryCatchList(expr, names[-nh], parentenv, handlers[-nh])
43: doTryCatch(return(expr), name, parentenv, handler)
44: tryCatchOne(tryCatchList(expr, names[-nh], parentenv, handlers[-nh]),     names[nh], parentenv, handlers[[nh]])
45: tryCatchList(expr, classes, parentenv, handlers)
46: tryCatch(withCallingHandlers({    NULL    saveRDS(do.call(do.call, c(readRDS("/tmp/RtmpKk0QhA/callr-fun-497929e2d840"),         list(envir = .GlobalEnv, quote = TRUE)), envir = .GlobalEnv,         quote = TRUE), file = "/tmp/RtmpKk0QhA/callr-res-49797239d88c",         compress = FALSE)    flush(stdout())    flush(stderr())    NULL    invisible()}, error = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpKk0QhA/callr-res-49797239d88c",             ".error"))    }}, interrupt = function(e) {    {        callr_data <- as.environment("tools:callr")$`__callr_data__`        err <- callr_data$err        if (FALSE) {            assign(".Traceback", .traceback(4), envir = callr_data)            dump.frames("__callr_dump__")            assign(".Last.dump", .GlobalEnv$`__callr_dump__`,                 envir = callr_data)            rm("__callr_dump__", envir = .GlobalEnv)        }        e <- err$process_call(e)        e2 <- err$new_error("error in callr subprocess")        class(e2) <- c("callr_remote_error", class(e2))        e2 <- err$add_trace_back(e2)        cut <- which(e2$trace$scope == "global")[1]        if (!is.na(cut)) {            e2$trace <- e2$trace[-(1:cut), ]        }        saveRDS(list("error", e2, e), file = paste0("/tmp/RtmpKk0QhA/callr-res-49797239d88c",             ".error"))    }}, callr_message = function(e) {    try(signalCondition(e))}), error = function(e) {    NULL    try(stop(e))}, interrupt = function(e) {    NULL    e})
An irrecoverable exception occurred. R is aborting now ...

This is the problem. 

> reprex::reprex({
+     greta:::have_tf()
+ },
+ si = TRUE,
+ std_out_err = TRUE)
njtierney commented 1 year ago

Hi there @BillyChen123

OK, so then to confirm things:

reprex::reprex(
  reticulate::py_module_available("tensorflow"), 
  si = TRUE,
  std_out_err = TRUE
  )

Does that produce the same error? In which case, you might be best off logging an issue with the team at tensorflow (here: https://github.com/rstudio/tensorflow) as it looks like the main issue is with installing tensorflow, and beyond the help that I've suggested here, I'm not sure how to help resolve this, unfortunately.

If you post an issue on the tensorflow issue page and link this issue by pasting this link, e.g., saying something like

Hi tensorflow team, I've been trying to debug installing tensorflow (as described here: https://github.com/greta-dev/greta/issues/565) and it looks like the issue is regarding installing a specific version of tensorflow with code like the following: reticulate::conda_install(envname = 'greta-env', packages = c('numpy==1.16.4', 'tensorflow-probability==0.7.0', 'tensorflow==1.14.0'))

Hopefully they can help from there? I'll keep a watch on the issue you open up!

njtierney commented 1 year ago

For what is it worth, I would try using the latest version of R to see if this problem persists, as using an older version of R makes the problem harder to reproduce for other users

BillyChen123 commented 1 year ago

Hi! @njtierney Thanks! I had post the issue on the tensorflow team, hopefully we can get the right answer.

njtierney commented 1 year ago

Thanks! I'll be keeping an eye on things over there. I'll close this issue for the time being.

t-kalinowski commented 1 year ago

@njtierney Is greta compatible with TF v2 today?

Looking at https://github.com/rstudio/tensorflow/issues/548, TF v1 is not something we include in our CI tests anymore or generally support, though if greta is still on TF v1 then that changes things.

If you are on TF v1, is there maybe a blocker I can help with on the tensorflow package side?