search

rstudio / keras3

R Interface to Keras
https://keras3.posit.co/
Other
833 stars 282 forks source link

AttributeError: 'tuple' object has no attribute 'layer' #880

Closed turgut090 closed 5 years ago

turgut090 commented 5 years ago

Hi, I just want to extract some layers from pretrained BERT within R. Actually, this process works fine for sequential models ( like adding layers or removing). But bert is very different..


inputs = model$inputs[1:2]
inputs 

[[1]]
Tensor("Input-Token:0", shape=(?, 300), dtype=float32)

[[2]]
Tensor("Input-Segment:0", shape=(?, 300), dtype=float32)

dense = get_layer(model,name = 'NSP-Dense') %>% get_output_at(1) 
dense

Tensor("NSP-Dense/Tanh:0", shape=(?, 768), dtype=float32)

Then I just want to add the following.

outputs = dense %>%  layer_dense(units=1, activation='sigmoid',
                                 kernel_initializer=initializer_truncated_normal(stddev = 0.02),
                                 name = 'real_output')
Error in py_call_impl(callable, dots$args, dots$keywords) : 
  AttributeError: 'tuple' object has no attribute 'layer'

Detailed traceback: 
  File "/Applications/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 881, in __call__
    inputs, outputs, args, kwargs)
  File "/Applications/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2043, in _set_connectivity_metadata_
    input_tensors=inputs, output_tensors=outputs, arguments=arguments)
  File "/Applications/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2059, in _add_inbound_node
    input_tensors)
  File "/Applications/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/Applications/anaconda3/lib/python3.7/site-packages/tensorflow_core/python/util/nest.py", line 536, in <listcomp>
    structure[0], [func(*x) for x in entries],
  File "/Appli

With python it is easy to do that:

inputs = model.inputs[:2]
dense = model.layers[-3].output
outputs = keras.layers.Dense(1, activation='sigmoid', kernel_initializer=keras.initializers.TruncatedNormal(stddev=0.02),
                             name = 'real_output')(dense)

Is this an object error? So, in R it is a list?

adilapapaya commented 4 years ago

@henry090 I'm facing this same error following the blog post which I think was a result of #53 above (https://blogs.rstudio.com/tensorflow/posts/2019-09-30-bert-r/). What was the resolution for fixing this?

My error is exactly the same as yours only I'm on Python 3.6

outputs = dense %>% layer_dense(units=1L, activation='sigmoid',
                         kernel_initializer=initializer_truncated_normal(stddev = 0.02),
                         name = 'output')

produces:

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  AttributeError: 'tuple' object has no attribute 'layer'

Detailed traceback: 
  File "/mnt/miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 874, in __call__
    inputs, outputs, args, kwargs)
  File "/mnt/miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2038, in _set_connectivity_metadata_
    input_tensors=inputs, output_tensors=outputs, arguments=arguments)
  File "/mnt/miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/base_layer.py", line 2054, in _add_inbound_node
    input_tensors)
  File "/mnt/miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py", line 535, in map_structure
    structure[0], [func(*x) for x in entries],
  File "/mnt/miniconda/envs/r-reticulate/lib/python3.6/site-packages/tensorflow_core/python/util/nest.py", line 535, in <listcomp>
    structure[0]
turgut090 commented 4 years ago

@adilapapaya Did you execute this Sys.setenv(TF_KERAS=1)? And could you share session info, please?

adilapapaya commented 4 years ago

@henry090 wow, that fixed it! I re-read the post and realized I completely missed that line. Thanks for the quick reply!

Here's my sessionInfo (I'm trying this out with tensorflow 2.0 and python 3.6 and so far there hasn't been an issue except for what I mentioned above)

# R 3.2.3 because that's the version on the gpu available to me and i haven't bothered to upgrade.
R version 3.2.3 (2015-12-10) 
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.5 LTS

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] keras_2.2.5.0    tensorflow_2.0.0 crayon_1.3.4     glue_1.3.1      
[5] dplyr_0.8.3     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.3        zeallot_0.1.0     assertthat_0.2.1  rappdirs_0.3.1   
 [5] R6_2.4.1          jsonlite_1.6      magrittr_1.5      pillar_1.4.3     
 [9] tfruns_1.4        rlang_0.4.2       data.table_1.12.8 whisker_0.4      
[13] generics_0.0.2    reticulate_1.14   purrr_0.3.3       pkgconfig_2.0.3  
[17] base64enc_0.1-3   tidyselect_0.2.5  tibble_2.1.3

To be more precise, my process was

  1. In terminal:

    # activate r-reticulate environment
conda activate r-reticulate

  2. In R:

    
# load all the libs I'm using
library("dplyr")
library("glue")
library("crayon")
library("tensorflow")
library("keras")

set TF_KERAS=1

Sys.setenv(TF_KERAS=1)

Do all the things from https://blogs.rstudio.com/tensorflow/posts/2019-09-30-bert-r/

daniela-winkler commented 3 years ago

I'm facing the same problem and already tried Sys.setenv(TF_KERAS=1) without success.

I'm using tensorflow 1.14, keras 2.2.4 and keras-bert 0.84. My sessionInfo looks like this:

R version 4.0.3 (2020-10-10)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)

Matrix products: default

locale:
[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252    LC_MONETARY=German_Germany.1252
[4] LC_NUMERIC=C                    LC_TIME=German_Germany.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] keras_2.3.0.0          reticulate_1.18        tensorflow_2.2.0       textshape_1.7.1       
 [5] corpus_0.10.1          qdap_2.4.3             RColorBrewer_1.1-2     qdapTools_1.3.5       
 [9] qdapRegex_0.7.2        qdapDictionaries_1.0.7 tm_0.7-8               NLP_0.2-1             
[13] readr_1.4.0            janitor_2.0.1          tidytext_0.2.6         textclean_0.9.3       
[17] dplyr_1.0.2            caret_6.0-86           ggplot2_3.3.2          lattice_0.20-41       
[21] jsonlite_1.7.1        

loaded via a namespace (and not attached):
 [1] nlme_3.1-149         bitops_1.0-6         lubridate_1.7.9.2    SnowballC_0.7.0      tools_4.0.3         
 [6] utf8_1.1.4           R6_2.5.0             rpart_4.1-15         colorspace_2.0-0     openNLPdata_1.5.3-4 
[11] nnet_7.3-14          withr_2.3.0          tidyselect_1.1.0     gridExtra_2.3        compiler_4.0.3      
[16] chron_2.3-56         xml2_1.3.2           slam_0.1-47          scales_1.1.1         rappdirs_0.3.1      
[21] tfruns_1.4           stringr_1.4.0        base64enc_0.1-3      pkgconfig_2.0.3      plotrix_3.7-8       
[26] rlang_0.4.8          rstudioapi_0.13      generics_0.1.0       openNLP_0.2-7        ModelMetrics_1.2.2.2
[31] zip_2.1.1            tokenizers_0.2.1     RCurl_1.98-1.2       magrittr_2.0.1       wordcloud_2.6       
[36] Matrix_1.2-18        Rcpp_1.0.5           munsell_0.5.0        lifecycle_0.2.0      stringi_1.5.3       
[41] whisker_0.4          pROC_1.16.2          snakecase_0.11.0     MASS_7.3-53          plyr_1.8.6          
[46] recipes_0.1.15       grid_4.0.3           parallel_4.0.3       gender_0.5.4         crayon_1.3.4        
[51] splines_4.0.3        hms_0.5.3            zeallot_0.1.0        venneuler_1.1-0      pillar_1.4.7        
[56] igraph_1.2.6         reshape2_1.4.4       codetools_0.2-16     stats4_4.0.3         XML_3.99-0.5        
[61] glue_1.4.2           data.table_1.13.2    vctrs_0.3.5          foreach_1.5.1        gtable_0.3.0        
[66] purrr_0.3.4          gower_0.2.2          openxlsx_4.2.3       prodlim_2019.11.13   janeaustenr_0.1.5   
[71] class_7.3-17         survival_3.2-7       timeDate_3043.102    tibble_3.0.4         rJava_0.9-13        
[76] iterators_1.0.13     lava_1.6.8.1         ellipsis_0.3.1       ipred_0.9-9

Does anybody know what to do?