Tensor Type error with keras3 package but not keras

[jonbry]

I have been getting a tensor type error message when building the model (model <- keras_model_sequenial()) with the keras3 package, but not when it's run with keras. I'm not sure what causes the issue, but I have been able to replicate the fix multiple times when it comes up. Anecdotally, it appears to happen the first time I run the code since opening Rstudio. Here's the error message from the Reuters example:

Error in py_call_impl(callable, call_args$unnamed, call_args$named) : 
  TypeError: Inputs to a layer should be tensors. Got '<keras.src.engine.sequential.Sequential object at 0x30bd7e320>' (of type <class 'keras.src.engine.sequential.Sequential'>) as input for layer 'dense_2’.

── Python Exception Message ──────────────────
Traceback (most recent call last):
  File "/Users/<user_account>/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/keras/src/utils/traceback_utils.py", line 70, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "/Users/<user_account>/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/keras/src/engine/input_spec.py", line 213, in assert_input_compatibility
    raise TypeError(
TypeError: Inputs to a layer should be tensors. Got '<keras.src.engine.sequential.Sequential object at 0x30bd7e320>' (of type <class 'keras.src.engine.sequential.Sequential'>) as input for layer 'dense_2'.

── R Traceback ───────────────────────────────
     ▆
  1. ├─... %>% layer_dense(46, activation = "softmax")
  2. ├─keras3::layer_dense(., 46, activation = "softmax")
  3. │ └─keras3:::create_layer(keras$layers$Dense, object, args)
  4. ├─keras3::layer_dense(., 64, activation = "relu")
  5. │ └─keras3:::create_layer(keras$layers$Dense, object, args)
  6. └─keras3::layer_dense(., 64, activation = "relu")
  7.   └─keras3:::create_layer(keras$layers$Dense, object, args)
  8.     └─keras3:::compose_layer(object, layer)
  9.       └─reticulate (local) layer(object, ...)
 10.         └─reticulate:::py_call_impl(callable, call_args$unnamed, call_args$named)

• Note: I changed the user account name to <user_account> in the error example

I don't believe this happens after each fresh boot of Rstudio, but it seems more likely to happen then. I copied the code from the repository to make sure it was correct and ran it with keras3.

I am able to fix the issue by changing the package to keras, and I get the notice that the follow S3 methods will be used by the keras package instead of keras3:

Registered S3 methods overwritten by 'keras':
  method                               from  
  as.data.frame.keras_training_history keras3
  plot.keras_training_history          keras3
  print.keras_training_history         keras3
  r_to_py.R6ClassGenerator             keras3

I then run the code and everything works fine. I can then run rm=list(ls) and rerun everything with keras3 and it still works fine, but I noticed that the environment for keras_model_sequential() is still listed as keras.

Let me know if there is any additional information I can provide the will be helpful.

[t-kalinowski]

Thanks for reporting!

Can you please provide an MRE? I suspect the issue is specific to R sessions where both keras and keras3 packages are loaded, due to conflicting S3 class names. (Specifically, keras registered a global S3 name filter, and keras3 doesn't).

Can you confirm/check that the issue doesn't occur if the keras package is not loaded?

[jonbry]

Thanks for getting back to me so quickly! Here's the reprex:

library(keras3)
reuters <- dataset_reuters(num_words = 10000)
c(c(train_data, train_labels), c(test_data, test_labels)) %<-% reuters

vectorize_sequences <- function(sequences, dimension = 10000) {
  results <- matrix(0, nrow = length(sequences), ncol = dimension)
  for (i in seq_along(sequences))
    results[i, sequences[[i]]] <- 1
  results
}

x_train <- vectorize_sequences(train_data)
x_test <- vectorize_sequences(test_data)

to_one_hot <- function(labels, dimension = 46) {
  results <- matrix(0, nrow = length(labels), ncol = dimension)
  labels <- labels + 1
  for(i in seq_along(labels)) {
    j <- labels[[i]]
    results[i, j] <- 1
  }
  results
}
y_train <- to_one_hot(train_labels)
y_test <- to_one_hot(test_labels)

model <- keras_model_sequential() %>%
  layer_dense(64, activation = "relu") %>%
  layer_dense(64, activation = "relu") %>%
  layer_dense(46, activation = "softmax")
#> Inputs to a layer should be tensors. Got
#> '<keras.src.engine.sequential.Sequential object at 0x17f77e620>' (of type
#> <class 'keras.src.engine.sequential.Sequential'>) as input for layer 'dense_2'.

model %>% compile(optimizer = "rmsprop",
                  loss = "categorical_crossentropy",
                  metrics = "accuracy")
#> Error in eval(expr, envir, enclos): object 'model' not found

^{Created on 2024-03-12 with reprex v2.1.0}

I ran this after opening RStudio and can confirm that the keras package was not loaded. I previously worked in a .qmd file, so I used an R script for the reprex just in case. Let me know if there is more information that I can provide for troubleshooting.

[t-kalinowski]

Hi, I can't reproduce the issue. This is what I see when I run the MRE.

library(keras3)
reuters <- dataset_reuters(num_words = 10000)
c(c(train_data, train_labels), c(test_data, test_labels)) %<-% reuters

vectorize_sequences <- function(sequences, dimension = 10000) {
  results <- matrix(0, nrow = length(sequences), ncol = dimension)
  for (i in seq_along(sequences))
    results[i, sequences[[i]]] <- 1
  results
}

x_train <- vectorize_sequences(train_data)
x_test <- vectorize_sequences(test_data)

to_one_hot <- function(labels, dimension = 46) {
  results <- matrix(0, nrow = length(labels), ncol = dimension)
  labels <- labels + 1
  for(i in seq_along(labels)) {
    j <- labels[[i]]
    results[i, j] <- 1
  }
  results
}
y_train <- to_one_hot(train_labels)
y_test <- to_one_hot(test_labels)

model <- keras_model_sequential() %>%
  layer_dense(64, activation = "relu") %>%
  layer_dense(64, activation = "relu") %>%
  layer_dense(46, activation = "softmax")

model %>% compile(optimizer = "rmsprop",
                  loss = "categorical_crossentropy",
                  metrics = "accuracy")

model
#> Model: "sequential"
#> ┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
#> ┃ Layer (type)                      ┃ Output Shape             ┃       Param # ┃
#> ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
#> │ dense_2 (Dense)                   │ ?                        │   0 (unbuilt) │
#> ├───────────────────────────────────┼──────────────────────────┼───────────────┤
#> │ dense_1 (Dense)                   │ ?                        │   0 (unbuilt) │
#> ├───────────────────────────────────┼──────────────────────────┼───────────────┤
#> │ dense (Dense)                     │ ?                        │   0 (unbuilt) │
#> └───────────────────────────────────┴──────────────────────────┴───────────────┘
#>  Total params: 0 (0.00 B)
#>  Trainable params: 0 (0.00 B)
#>  Non-trainable params: 0 (0.00 B)

^{Created on 2024-03-12 with reprex v2.1.0}

Can you please post the output from sessioninfo::session_info() and reticulate::py_list_packages()?

[jonbry]

Well, of course it now runs ok. The only changes since I last ran it was updating tensorflow to 2.16.1. Is it possible that fixed the issue?

[t-kalinowski]

My best guess is that this inherits() call in keras3 returns FALSE because of this global filter registered by keras.

[jonbry]

Here's the session and package info:

 setting  value
 version  R version 4.3.2 (2023-10-31)
 os       macOS Sonoma 14.4
 system   aarch64, darwin20
 ui       RStudio
 language (EN)
 collate  en_US.UTF-8
 ctype    en_US.UTF-8
 tz       America/New_York
 date     2024-03-12
 rstudio  2023.12.0+369 Ocean Storm (desktop)
 pandoc   NA

─ Packages ──────────────────────────────────
 package     * version     date (UTC) lib source
 base64enc     0.1-3       2015-07-28 [1] CRAN (R 4.3.0)
 cli           3.6.2       2023-12-11 [1] CRAN (R 4.3.1)
 generics      0.1.3       2022-07-05 [1] CRAN (R 4.3.0)
 glue          1.7.0       2024-01-09 [1] CRAN (R 4.3.1)
 here          1.0.1       2020-12-13 [1] CRAN (R 4.3.0)
 jsonlite      1.8.8       2023-12-04 [1] CRAN (R 4.3.1)
 keras3      * 0.1.0       2024-02-17 [1] CRAN (R 4.3.1)
 lattice       0.21-9      2023-10-01 [1] CRAN (R 4.3.2)
 lifecycle     1.0.4       2023-11-07 [1] CRAN (R 4.3.1)
 magrittr      2.0.3       2022-03-30 [1] CRAN (R 4.3.0)
 Matrix        1.6-1.1     2023-09-18 [1] CRAN (R 4.3.2)
 png           0.1-8       2022-11-29 [1] CRAN (R 4.3.0)
 Rcpp          1.0.12      2024-01-09 [1] CRAN (R 4.3.1)
 reticulate    1.35.0      2024-01-31 [1] CRAN (R 4.3.1)
 rlang         1.1.3       2024-01-10 [1] CRAN (R 4.3.1)
 rprojroot     2.0.4       2023-11-05 [1] CRAN (R 4.3.1)
 rstudioapi    0.15.0      2023-07-07 [1] CRAN (R 4.3.0)
 sessioninfo   1.2.2       2021-12-06 [1] CRAN (R 4.3.0)
 tensorflow    2.15.0.9000 2024-02-29 [1] Github (rstudio/tensorflow@15ee3a1)
 tfruns        1.5.2       2024-01-26 [1] CRAN (R 4.3.1)
 whisker       0.4.1       2022-12-05 [1] CRAN (R 4.3.0)
 withr         3.0.0       2024-01-16 [1] CRAN (R 4.3.1)
 zeallot       0.1.0       2018-01-28 [1] CRAN (R 4.3.0)

 [1] /Library/Frameworks/R.framework/Versions/4.3-arm64/Resources/library

─ Python configuration ──────────────────────
 python:         /Users/<user_name>/.virtualenvs/r-tensorflow/bin/python
 libpython:      /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/config-3.10-darwin/libpython3.10.dylib
 pythonhome:     /Users/<user_name>/.virtualenvs/r-tensorflow:/Users/<user_name>/.virtualenvs/r-tensorflow
 version:        3.10.4 (v3.10.4:9d38120e33, Mar 23 2022, 17:29:05) [Clang 13.0.0 (clang-1300.0.29.30)]
 numpy:          /Users/<user_name>/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/numpy
 numpy_version:  1.26.4
 keras:          /Users/<user_name>/.virtualenvs/r-tensorflow/lib/python3.10/site-packages/keras

 NOTE: Python version was forced by import("keras")

─────────────────────────────────────────────
> reticulate::py_list_packages()
                        package  version
1                       absl-py    2.1.0
2                    astunparse    1.6.3
3                       certifi 2024.2.2
4            charset-normalizer    3.3.2
5                       dm-tree    0.1.8
6                   flatbuffers   24.3.7
7                          gast    0.5.4
8                  google-pasta    0.2.0
9                        grpcio   1.62.1
10                         h5py   3.10.0
11                         idna      3.6
12                        keras    3.0.5
13                     libclang   16.0.6
14                     Markdown    3.5.2
15               markdown-it-py    3.0.0
16                   MarkupSafe    2.1.5
17                        mdurl    0.1.2
18                    ml-dtypes    0.3.2
19                        namex    0.0.7
20                        numpy   1.26.4
21                   opt-einsum    3.3.0
22                    packaging     24.0
23                     protobuf   4.25.3
24                     Pygments   2.17.2
25                     requests   2.31.0
26                         rich   13.7.1
27                          six   1.16.0
28                  tensorboard   2.16.2
29      tensorboard-data-server    0.7.2
30                   tensorflow   2.16.1
31 tensorflow-io-gcs-filesystem   0.36.0
32             tensorflow-metal    1.1.0
33                    termcolor    2.4.0
34            typing_extensions   4.10.0
35                      urllib3    2.2.1
36                     Werkzeug    3.0.1
37                        wrapt   1.16.0
                            requirement
1                        absl-py==2.1.0
2                     astunparse==1.6.3
3                     certifi==2024.2.2
4             charset-normalizer==3.3.2
5                        dm-tree==0.1.8
6                   flatbuffers==24.3.7
7                           gast==0.5.4
8                   google-pasta==0.2.0
9                        grpcio==1.62.1
10                         h5py==3.10.0
11                            idna==3.6
12                         keras==3.0.5
13                     libclang==16.0.6
14                      Markdown==3.5.2
15                markdown-it-py==3.0.0
16                    MarkupSafe==2.1.5
17                         mdurl==0.1.2
18                     ml-dtypes==0.3.2
19                         namex==0.0.7
20                        numpy==1.26.4
21                    opt-einsum==3.3.0
22                      packaging==24.0
23                     protobuf==4.25.3
24                     Pygments==2.17.2
25                     requests==2.31.0
26                         rich==13.7.1
27                          six==1.16.0
28                  tensorboard==2.16.2
29       tensorboard-data-server==0.7.2
30                   tensorflow==2.16.1
31 tensorflow-io-gcs-filesystem==0.36.0
32              tensorflow-metal==1.1.0
33                     termcolor==2.4.0
34            typing_extensions==4.10.0
35                       urllib3==2.2.1
36                      Werkzeug==3.0.1
37                        wrapt==1.16.0

It says I'm using tensorflow 2.15.0.9000, but I know I just updated to 2.16.1. Should I mark the issues as complete now that it runs?

[t-kalinowski]

I think this is related to the global S3 class filter registered by keras. I'll close this issue once we update keras so that both keras and keras3 can be safely loaded in the same R session.

[jonbry]

I saw that you just released keras 2.15.0 that allows for both keras and keras3 to be loaded together. Can I just create a new virtualenv (something like install_keras(envname = "keras2", backend = "tensorflow") to be able to run both keras and keras3 code?

I'm still having an issue with one of the models with keras3 0.2.0 and would like to run it with keras to make sure I'm not crazy.

Thank you!

[t-kalinowski]

The {keras} package will default to using the r-tensorflow venv, so you can have both installed alongside each other

keras::install_keras() # create r-tensorflow venv, with TF v2.15 (which bundled Keras 2)
keras3::install_keras() # create r-keras venv using the latest TF and Keras 3

[jonbry]

Awesome! Thanks for getting back to me so quickly!