Closed jobstdavid closed 6 months ago
t-kalinowski
commented
6 months ago Very interesting. Thanks for reporting.
Do you get the same error if you use keras3?
library(keras3)
keras_model_sequential() What do the broken /usr/local/bin/python* links point to?
system("ls -l /usr/local/bin/python*")Can you trackdown where the error is coming from? Something like this might help:
options(error = quote(print(rlang::trace_back())))
<reproduce error>
jobstdavid
commented
6 months ago Thanks, with keras3 it works without any problem! What is the (main) difference between keras3 and keras?
Furthermore, I receive
system("ls -l /usr/local/bin/python*")
lrwxr-xr-x 1 root admin 70 Nov 24 2021 /usr/local/bin/python3 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3
lrwxr-xr-x 1 root admin 77 Nov 24 2021 /usr/local/bin/python3-config -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3-config
lrwxr-xr-x 1 root admin 78 Nov 24 2021 /usr/local/bin/python3-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3-intel64
lrwxr-xr-x 1 root admin 73 Nov 24 2021 /usr/local/bin/python3.10 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10
lrwxr-xr-x 1 root admin 80 Nov 24 2021 /usr/local/bin/python3.10-config -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-config
lrwxr-xr-x 1 root admin 81 Nov 24 2021 /usr/local/bin/python3.10-intel64 -> ../../../Library/Frameworks/Python.framework/Versions/3.10/bin/python3.10-intel64and
options(error = quote(print(rlang::trace_back())))
model <- keras_model_sequential()
Error:
▆Thanks for reporting, the warning about No such file or directory is fixed in the development version of reticulate now, but that doesn't fix the error you see from keras.
We'll need a traceback to track it down---I can't reproduce the error locally.
Here is another trick for getting a traceback if R is being "shy" like this, wrap the offending call in a function that you're tracing in a debugger, and then repeatedly "step into" (s and Enter, or Shift + F4 in the Rstudio IDE) until you observe the error. At anytime, you can print print(rlang::trace_back()) to see a pretty printout of the current stack trace.
Seeing as how you have some broken symlinks in /usr/local/bin/*, my guess is that there might be other such artifacts related to former python installations on the system, and that might ultimately be the root cause here. Perhaps re-installing python and then recreating the venv might fix the issue in that case.
reticulate::install_python("3.10:latest", force = TRUE) -> python
# TensorFlow version 2.15 is the last version that defaulted to using Keras version 2,
# which is what the {keras} package is for. TensorFlow 2.16 and later default
# to Keras 3, which you can use from R using the {keras3} package.
tensorflow::install_tensorflow(version = "2.15", python_version = python)Also, please double check that PYTHONPATH is not set, nor that any conda hooks are automatically activated in .zshrc or .bashrc or similar.
is_set <- function(x) !is.na(Sys.getenv(x, NA))
if (is_set(c("PYTHONPATH")))
stop("PYTHONPATH is being set somewhere")
if (any(is_set(c("CONDA_PREFIX", "CONDA_EXE"))))
stop("Conda is being activated somewhere")Thanks for the information!
I tried the following in R to trace the error:
profvis::profvis({
keras::keras_model_sequential()
})
profvis: code exited with error:
Error in parse_rprof_lines(lines, expr_source) :
No parsing data available. Maybe your function was too fast?I don't think profvis will be helpful here. I meant to do something like this
fn <- function() {
browser()
keras::keras_model_sequential()
}
fn()
# press the 's' key and then the Enter key
# press the 's' key and then the Enter key
# press the 's' key and then the Enter key
# press the 's' key and then the Enter key
# ... see where we're going
# evaluate `print(rlang::trace_back())` to see the tracebackThanks! That's the output:
> fn <- function() {
+ browser()
+ keras::keras_model_sequential()
+ }
>
> fn()
Called from: fn()
Browse[1]> s
debug at #3: keras::keras_model_sequential()
Browse[2]> s
debugging in: keras::keras_model_sequential()
debug: {
if (length(list(...)))
layers <- c(sequential_model_input_layer(...), layers)
keras$models$Sequential(layers = layers, name = name)
}
Browse[3]> s
debug: if (length(list(...))) layers <- c(sequential_model_input_layer(...),
layers)
Browse[3]> s
debug: keras$models$Sequential(layers = layers, name = name)
Browse[3]> s
debugging in: keras$models$Sequential(layers = layers, name = name)
debug: {
cl <- sys.call()
cl[[1L]] <- list2
call_args <- split_named_unnamed(eval(cl, parent.frame()))
result <- py_call_impl(callable, call_args$unnamed, call_args$named)
if (py_get_convert(callable))
result <- py_to_r(result)
if (is.null(result))
invisible(result)
else result
}
Browse[4]> s
debug: cl <- sys.call()
Browse[4]> s
debugging in: sys.call()
debug: .Internal(sys.call(which))
Browse[5]> s
exiting from: sys.call()
debug: cl[[1L]] <- list2
Browse[4]> s
debug: call_args <- split_named_unnamed(eval(cl, parent.frame()))
Browse[4]> s
debugging in: split_named_unnamed(eval(cl, parent.frame()))
debug: {
nms <- names(x)
if (is.null(nms))
return(list(unnamed = x, named = list()))
named <- nzchar(nms)
list(unnamed = x[!named], named = x[named])
}
Browse[5]> s
debug: nms <- names(x)
Browse[5]> s
debugging in: eval(cl, parent.frame())
debug: .Internal(eval(expr, envir, enclos))
Browse[6]> s
debugging in: parent.frame()
debug: .Internal(parent.frame(n))
Browse[7]> s
exiting from: parent.frame()
debug: baseenv()
Browse[6]> s
debugging in: (function (...)
{
.Call(ffi_dots_list, frame_env = environment(), named = NULL,
ignore_empty = "trailing", preserve_empty = FALSE, unquote_names = TRUE,
homonyms = "keep", check_assign = FALSE)
})(layers = layers, name = name)
debug: {
.Call(ffi_dots_list, frame_env = environment(), named = NULL,
ignore_empty = "trailing", preserve_empty = FALSE, unquote_names = TRUE,
homonyms = "keep", check_assign = FALSE)
}
Browse[8]> s
debug: .Call(ffi_dots_list, frame_env = environment(), named = NULL,
ignore_empty = "trailing", preserve_empty = FALSE, unquote_names = TRUE,
homonyms = "keep", check_assign = FALSE)
Browse[8]> s
debugging in: environment()
debug: .Internal(environment(fun))
Browse[9]> s
exiting from: environment()
exiting from: (function (...)
{
.Call(ffi_dots_list, frame_env = environment(), named = NULL,
ignore_empty = "trailing", preserve_empty = FALSE, unquote_names = TRUE,
homonyms = "keep", check_assign = FALSE)
})(layers = layers, name = name)
exiting from: eval(cl, parent.frame())
debug: if (is.null(nms)) return(list(unnamed = x, named = list()))
Browse[5]> s
debug: named <- nzchar(nms)
Browse[5]> s
debug: list(unnamed = x[!named], named = x[named])
Browse[5]> s
exiting from: split_named_unnamed(eval(cl, parent.frame()))
debug: result <- py_call_impl(callable, call_args$unnamed, call_args$named)
Browse[4]> s
debugging in: py_call_impl(callable, call_args$unnamed, call_args$named)
debug: {
.Call(`_reticulate_py_call_impl`, x, args, keywords)
}
Browse[5]> s
debug: .Call(`_reticulate_py_call_impl`, x, args, keywords)
Browse[5]> s
exiting from: py_call_impl(callable, call_args$unnamed, call_args$named)
debug: if (py_get_convert(callable)) result <- py_to_r(result)
Browse[4]> s
debugging in: py_get_convert(callable)
debug: {
.Call(`_reticulate_py_get_convert`, x)
}
Browse[5]> s
debug: .Call(`_reticulate_py_get_convert`, x)
Browse[5]> s
exiting from: py_get_convert(callable)
debug: result <- py_to_r(result)
Browse[4]> s
debugging in: py_to_r(result)
debug: {
if (!is_py_object(x <- py_to_r_cpp(x)))
return(x)
UseMethod("py_to_r")
}
Browse[5]> s
debug: if (!is_py_object(x <- py_to_r_cpp(x))) return(x)
Browse[5]> s
debugging in: is_py_object(x <- py_to_r_cpp(x))
debug: {
.Call(`_reticulate_is_py_object`, x)
}
Browse[6]> s
debug: .Call(`_reticulate_is_py_object`, x)
Browse[6]> s
debugging in: py_to_r_cpp(x)
debug: {
.Call(`_reticulate_py_to_r_cpp`, x)
}
Browse[7]> s
debug: .Call(`_reticulate_py_to_r_cpp`, x)
Browse[7]> s
exiting from: py_to_r_cpp(x)
exiting from: is_py_object(x <- py_to_r_cpp(x))
debug: UseMethod("py_to_r")
Browse[5]> s
debugging in: py_to_r.default(result)
debug: {
py_to_r_cpp(x)
}
Browse[6]> s
debug: py_to_r_cpp(x)
Browse[6]> s
debugging in: py_to_r_cpp(x)
debug: {
.Call(`_reticulate_py_to_r_cpp`, x)
}
Browse[7]> s
debug: .Call(`_reticulate_py_to_r_cpp`, x)
Browse[7]> s
exiting from: py_to_r_cpp(x)
exiting from: py_to_r.default(result)
exiting from: py_to_r(result)
debug: if (is.null(result)) invisible(result) else result
Browse[4]> s
debug: result
Browse[4]> s
exiting from: keras$models$Sequential(layers = layers, name = name)
exiting from: keras::keras_model_sequential()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Layer (type) ┃ Output Shape ┃ Param # ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
└─────────────────────────────────────────────────────────────────────────────────────┴─────────────────────────────────────────────────────────────────┴──────────────────────────────────────┘
Total params: 0 (0.00 B)
Trainable params: 0 (0.00 B)
Non-trainable params: 0 (0.00 B)it seems that there is no error when you step through it with a debugger. Am I reading the output correctly?
But the error still occurs outside a debugger?
jobstdavid
commented
6 months ago Yes, you are reading the output correctly. During debugging, the error does not occur, but outside the debugging the error does occur.
t-kalinowski
commented
6 months ago I'm genuinely stumped about what might be causing the issue.
Do you have anything in .Rprofile or .Renviron that might be causing this?
Any other packages loaded?
jobstdavid
commented
6 months ago Wow, thank you very much!!! Indeed, there was an exported path in the .Rprofil linking to a conda env, which I deleted and now it works!!! Thank you so much for your great support!
I followed you installation instructions and receive the following error messages:
Some diagnostic info:
Unfortunately, calling the code on
https://tensorflow.rstudio.com/examples/conv_lstm
yields to an error as well:
(There is no explicit error message)
Thank you very much in advance!