search

rstudio / keras3

R Interface to Keras
https://keras3.posit.co/
Other
833 stars 282 forks source link

Siamese Networks in R with Keras lead to RuntimeError(res[kErrorKey]) #1042

Closed GMTSE closed 4 years ago

GMTSE commented 4 years ago

Hi,

Thanks for your work on Keras for R, it is very useful!

Context: I'm trying to implement Siamese Networks in R+library(keras).

Of course, the first thing I imagined was to start simple, by working on something like MNIST-digit data. Also, before trying something myself, I wanted to see if there was a simple working example to test if my environment was OK. I found this simple code in Python that works well and that I understand well: https://github.com/rstudio/keras/blob/master/vignettes/examples/mnist_siamese_graph.py Unfortunately, it's equivalent in R is empty: https://github.com/rstudio/keras/blob/master/vignettes/examples/mnist_siamese_graph.R So I decided to translate the code, so that I can learn by doing and also I could contribute to the project in the end if the result is satisfying. The translation is not finished, but here is what I got for the moment: SiameseNetworks.txt (As can be noted, I modified a little the couples' generation, to work only on the indices first, then extract the images, because the original code, that [ac]binding pair by pair to the final data structure takes forever on my machine).

Everything works fine until I try to fit the model... I get the following error:

Error in py_call_impl(callable, dots$args, dots$keywords) :
  RuntimeError: Evaluation error: invalid argument type.

Detailed traceback:
  File "/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 66, in _method_wrapper
    return method(self, *args, **kwargs)
  File "/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 830, in fit
    callbacks.on_train_begin()
  File "/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py", line 447, in on_train_begin
    callback.on_train_begin(logs)
  File "/home/user/.R/x86_64-pc-linux-gnu-library/3.6/keras/python/kerastools/callback.py", line 57, in on_train_begin
    self.r_on_train_begin(logs)
  File "/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/python/rpytools/call.py", line 21, in python_function
    raise RuntimeError(res[kErrorKey])

In more details:

17:00:51 R > traceback()
14: stop(list(message = "RuntimeError: Evaluation error: invalid argument type.\n\nDetailed traceback: \n  File \"/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py\", line 66, in _method_wrapper\n    return method(self, *args, **kwargs)\n  File \"/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py\", line 830, in fit\n    callbacks.on_train_begin()\n  File \"/home/user/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow/python/keras/callbacks.py\", line 447, in on_train_begin\n    callback.on_train_begin(logs)\n  File \"/home/user/.R/x86_64-pc-linux-gnu-library/3.6/keras/python/kerastools/callback.py\", line 57, in on_train_begin\n    self.r_on_train_begin(logs)\n  File \"/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/python/rpytools/call.py\", line 21, in python_function\n    raise RuntimeError(res[kErrorKey])\n",
        call = py_call_impl(callable, dots$args, dots$keywords),
        cppstack = list(file = "", line = -1L, stack = c("/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/libs/reticulate.so(Rcpp::exception::exception(char const*, bool)+0x74) [0x7f50f05cb274]",
        "/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/libs/reticulate.so(Rcpp::stop(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x29) [0x7f50f05bba46]",
        "/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/libs/reticulate.so(+0x1b5b6) [0x7f50f05be5b6]",
        "/home/user/.R/x86_64-pc-linux-gnu-library/3.6/reticulate/libs/reticulate.so(_reticulate_py_call_impl+0xa9) [0x7f50f05c6bb9]",
        "/usr/lib/R/lib/libR.so(+0x12f557) [0x7f5103088557]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x2ba) [0x7f510309c72a]",
        "/usr/lib/R/lib/libR.so(+0xbe1cd) [0x7f51030171cd]", "/usr/lib/R/lib/libR.so(+0x1276d0) [0x7f51030806d0]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x1856e3) [0x7f51030de6e3]", "/usr/lib/R/lib/libR.so(+0x185b10) [0x7f51030deb10]",
        "/usr/lib/R/lib/libR.so(+0x185ec6) [0x7f51030deec6]", "/usr/lib/R/lib/libR.so(+0x128df0) [0x7f5103081df0]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x2ba) [0x7f510309c72a]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x143f3c) [0x7f510309cf3c]", "/usr/lib/R/lib/libR.so(Rf_eval+0x83e) [0x7f510309ccae]",
        "/usr/lib/R/lib/libR.so(+0x149825) [0x7f51030a2825]", "/usr/lib/R/lib/libR.so(+0x1834e9) [0x7f51030dc4e9]",
        "/usr/lib/R/lib/libR.so(+0x128df0) [0x7f5103081df0]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x2ba) [0x7f510309c72a]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x2ba) [0x7f510309c72a]",
        "/usr/lib/R/lib/libR.so(+0x148d32) [0x7f51030a1d32]", "/usr/lib/R/lib/libR.so(+0x1276d0) [0x7f51030806d0]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x143f3c) [0x7f510309cf3c]", "/usr/lib/R/lib/libR.so(Rf_eval+0x83e) [0x7f510309ccae]",
        "/usr/lib/R/lib/libR.so(+0x149825) [0x7f51030a2825]", "/usr/lib/R/lib/libR.so(+0x1834e9) [0x7f51030dc4e9]",
        "/usr/lib/R/lib/libR.so(+0x128df0) [0x7f5103081df0]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(+0x135405) [0x7f510308e405]", "/usr/lib/R/lib/libR.so(Rf_eval+0x1a0) [0x7f510309c610]",
        "/usr/lib/R/lib/libR.so(+0x14541f) [0x7f510309e41f]", "/usr/lib/R/lib/libR.so(Rf_applyClosure+0x163) [0x7f510309f253]",
        "/usr/lib/R/lib/libR.so(Rf_eval+0x2ba) [0x7f510309c72a]",
        "/usr/lib/R/lib/libR.so(Rf_ReplIteration+0x1f2) [0x7f51030ce1e2]",
        "/usr/lib/R/lib/libR.so(+0x175570) [0x7f51030ce570]", "/usr/lib/R/lib/libR.so(run_Rmainloop+0x50) [0x7f51030ce630]",
        "/usr/lib/rstudio/bin/rsession(rstudio::r::session::runEmbeddedR(rstudio::core::FilePath const&, rstudio::core::FilePath const&, bool, bool, SA_TYPE, rstudio::r::session::Callbacks const&, rstudio::r::session::InternalCallbacks*)+0x1b2) [0x55d4d73fae52]",
        "/usr/lib/rstudio/bin/rsession(rstudio::r::session::run(rstudio::r::session::ROptions const&, rstudio::r::session::RCallbacks const&)+0x1385) [0x55d4d73dd255]",
        "/usr/lib/rstudio/bin/rsession(main+0x2976) [0x55d4d6ce5d46]",
        "/usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f510267e0b3]",
        "/usr/lib/rstudio/bin/rsession(_start+0x2a) [0x55d4d6cf716a]"
        ))))
13: py_call_impl(callable, dots$args, dots$keywords)
12: (function (...)
    {
        dots <- py_resolve_dots(list(...))
        result <- py_call_impl(callable, dots$args, dots$keywords)
        if (convert)
            result <- py_to_r(result)
        if (is.null(result))
            invisible(result)
        else result
    })(batch_size = 32L, epochs = 10L, verbose = 1L, callbacks = list(
        <environment>), validation_split = 0, shuffle = TRUE, class_weight = NULL,
        sample_weight = NULL, initial_epoch = 0L, x = list(<environment>,
            <environment>), y = <environment>)
11: do.call(object$fit, args)
10: fit.keras.engine.training.Model(., x = list(train_left, train_right),
        y = train_labl)
9: fit(., x = list(train_left, train_right), y = train_labl)
8: function_list[[k]](value)
7: withVisible(function_list[[k]](value))
6: freduce(value, `_function_list`)
5: `_fseq`(`_lhs`)
4: eval(quote(`_fseq`(`_lhs`)), env, env)
3: eval(quote(`_fseq`(`_lhs`)), env, env)
2: withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))
1: model %>% fit(x = list(train_left, train_right), y = train_labl)

I've tried to simplify the code so instead of comparing images of digits, I directly compare the digits themselves (i.e. the input_a and input_b layer consist in a single digit). But I get the same error.

I've search the web for other examples, the only one I found that was as simpe as I needed (*) was in the book "Deep Learning with R" by Francois Chollet and J. J. Allaire (MEAP). I added a few data to run it against, and obtained the following very simple code: SiameseNetworks_Cholet.txt But got again the same error...

I've search a lot for this error and I can't find anything apart that it seems related to the fact the I provide a list as inputs when calling fit(). Reference: https://github.com/keras-team/keras/issues/13827

So I'm starting to think there is a bug in the keras lib. Can you confirm/infirm?

Thanks & stay safe! (*) I've found this one: https://www.kaggle.com/shih0430/mnist-siamese-neural-network-for-keras-r-code But the data generation is too complicated and I don't see a difference in the model part. EDIT: I tried this code and I also get the error

FYI:

> keras:::implementation()
Module(tensorflow.python.keras)
> tensorflow::tf_config()
TensorFlow v2.2.0 ()
Python v3.6 (~/.local/share/r-miniconda/envs/r-reticulate/bin/python)
> sessionInfo()
R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
LAPACK: /home/gmuller/.local/share/r-miniconda/envs/r-reticulate/lib/libmkl_rt.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] abind_1.4-5   permute_0.9-5 dplyr_0.8.5   keras_2.2.5.0

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.4.6     rstudioapi_0.11  whisker_0.4      magrittr_1.5     rappdirs_0.3.1   tidyselect_1.1.0 lattice_0.20-40  R6_2.4.1         rlang_0.4.6      tools_3.6.3      grid_3.6.3      
[12] ellipsis_0.3.1   tfruns_1.4       assertthat_0.2.1 tibble_3.0.1     lifecycle_0.2.0  crayon_1.3.4     tensorflow_2.2.0 Matrix_1.2-18    purrr_0.3.4      base64enc_0.1-3  vctrs_0.3.0     
[23] zeallot_0.1.0    glue_1.4.1       compiler_3.6.3   pillar_1.4.4     generics_0.0.2   reticulate_1.15  jsonlite_1.6.1   pkgconfig_2.0.3 
15:46:19 R > 
Linux empcarlabhc144 5.4.0-29-generic #33-Ubuntu SMP Wed Apr 29 14:32:27 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"
> reticulate::py_config()
python:         /home/gmuller/.local/share/r-miniconda/envs/r-reticulate/bin/python
libpython:      /home/gmuller/.local/share/r-miniconda/envs/r-reticulate/lib/libpython3.6m.so
pythonhome:     /home/gmuller/.local/share/r-miniconda/envs/r-reticulate:/home/gmuller/.local/share/r-miniconda/envs/r-reticulate
version:        3.6.10 |Anaconda, Inc.| (default, Jan  7 2020, 21:14:29)  [GCC 7.3.0]
numpy:          /home/gmuller/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages/numpy
numpy_version:  1.18.1
16:04:51 R > system("pip list")
Package                Version            
---------------------- -------------------
absl-py                0.9.0              
astor                  0.8.1              
astunparse             1.6.3              
backports.weakref      1.0rc1             
bleach                 1.5.0              
cachetools             4.0.0              
certifi                2020.4.5.1         
chardet                3.0.4              
gast                   0.3.3              
google-auth            1.11.0             
google-auth-oauthlib   0.4.1              
google-pasta           0.1.8              
grpcio                 1.27.1             
h5py                   2.10.0             
html5lib               0.9999999          
idna                   2.8                
Keras                  2.3.1              
Keras-Applications     1.0.8              
Keras-Preprocessing    1.1.0              
Markdown               3.2                
mkl-fft                1.0.15             
mkl-random             1.1.0              
mkl-service            2.3.0              
numpy                  1.18.1             
oauthlib               3.1.0              
opt-einsum             3.1.0              
Pillow                 7.1.2              
pip                    20.0.2             
protobuf               3.11.3             
pyasn1                 0.4.8              
pyasn1-modules         0.2.8              
PyYAML                 5.3.1              
requests               2.23.0             
requests-oauthlib      1.3.0              
rsa                    4.0                
scipy                  1.4.1              
setuptools             45.1.0.post20200127
six                    1.14.0             
tensorboard            2.2.1              
tensorboard-plugin-wit 1.6.0.post3        
tensorflow             2.2.0              
tensorflow-estimator   2.2.0              
tensorflow-hub         0.8.0              
termcolor              1.1.0              
urllib3                1.25.8             
Werkzeug               1.0.0              
wheel                  0.34.2             
wrapt                  1.11.2 
R > packageVersion("tensorflow")
[1] ‘2.2.0’
> packageVersion("keras")
[1] ‘2.2.5.0’
> packageVersion("reticulate")
[1] ‘1.15’
skeydan commented 4 years ago

Hi,

can you try installing the very newest tensorflow and keras from CRAN?

install.packages("tensorflow")
install.packages("keras")
GMTSE commented 4 years ago

Thanks for your quick response! Sorry I was completing my answer when you responded I'm currently installing the new versions. I'll tell you the result when it's done.

GMTSE commented 4 years ago

Hello again,

I managed to get my code running using

remotes::install_github("rstudio/reticulate")
remotes::install_github("rstudio/tensorflow")
remotes::install_github("rstudio/keras")
library(reticulate)
library(tensorflow)
library(keras)

From: https://github.com/rstudio/reticulate/issues/724#issuecomment-619100262

Thanks a lot for your help!