Error while running RStudio unet_linux example: AttributeError: 'RGenerator' object has no attribute 'shape'

gontcharovd commented 5 years ago

Here is the code I run: https://keras.rstudio.com/articles/examples/unet_linux.html

In the same folder as the script with the code I have a folder input with two subfolders train and train_masks. Both subfolders contain 5.088 images and masks respectively downloaded from https://www.kaggle.com/c/carvana-image-masking-challenge, as instructed in the example.

I get the following error when I execute the code:

model %>% fit_generator( 
  train_iterator,  
  steps_per_epoch = as.integer(length(train_index) / batch_size),   
  epochs = epochs, 
  validation_data = val_iterator,
  validation_steps = as.integer(length(val_index) / batch_size),
  verbose = 1,
  callbacks = callbacks_list
)

Error in py_call_impl(callable, dots$args, dots$keywords) : AttributeError: 'RGenerator' object has no attribute 'shape' Detailed traceback: File "/home/gontcharovd/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/training.py", line 1433, in fit_generator steps_name='steps_per_epoch') File "/home/gontcharovd/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_generator.py", line 144, in model_iteration shuffle=shuffle) File "/home/gontcharovd/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow/python/keras/engine/training_generator.py", line 480, in convert_to_generator_like num_samples = int(nest.flatten(data)[0].shape[0])`

This is what I have tried:

reinstalling R's reticulate package
reinstalling Keras with install_keras(tensorflow = "gpu")

Additional small problems:

There was one more small error at line 389: the function callback_reduce_lr_on_plateau has no argument epsilon.

Error in callback_reduce_lr_on_plateau(monitor = "val_python_function", : unused argument (epsilon = 1e-04)

Commenting out this line 389 resolves this error.

there are two warnings (but I prefer to handle the error above first):

WARNING:tensorflow:batch_size is no longer needed in the TensorBoard Callback and will be ignored in TensorFlow 2.0. WARNING:tensorflow:period argument is deprecated. Please use save_freq to specify the frequency in number of samples seen.

My environment:

> SessionInfo() 

R version 3.6.1 (2019-07-05)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.3 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1
LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8
 [7] LC_PAPER=de_DE.UTF-8       LC_NAME=C                  LC_ADDRESS=C
[10] LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods   base

other attached packages:
[1] doMC_1.3.6             iterators_1.0.12       foreach_1.4.7
[4] reticulate_1.13.0-9000 abind_1.4-5            magick_2.2
[7] keras_2.2.4.1

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.2        codetools_0.2-16  ps_1.3.0          zeallot_0.1.0
 [5] R6_2.4.0          jsonlite_1.6      magrittr_1.5      tfruns_1.4
 [9] rlang_0.4.0       whisker_0.3-2     generics_0.0.2    tools_3.6.1
[13] purrr_0.3.2       processx_3.4.1    compiler_3.6.1    base64enc_0.1-3
[17] tensorflow_1.14.0

> tensorflow::tf_config() 
TensorFlow v1.14.0 (~/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow)
Python v3.7 (~/.virtualenvs/r-reticulate/bin/python)

(in Ubuntu terminal) ~$ conda-env list
# conda environments:
#
base                *   /home/gontcharovd/anaconda3
tf-gpu                  /home/gontcharovd/anaconda3/envs/tf-gpu

> reticulate::py_config()
python:         /home/gontcharovd/.virtualenvs/r-reticulate/bin/python
libpython:      /home/gontcharovd/anaconda3/lib/libpython3.7m.so
pythonhome:     /home/gontcharovd/anaconda3:/home/gontcharovd/anaconda3
version:        3.7.3 (default, Mar 27 2019, 22:11:17)  [GCC 7.3.0]
numpy:          /home/gontcharovd/.virtualenvs/r-reticulate/lib/python3.7/site-packages/numpy
numpy_version:  1.17.1
tensorflow:     /home/gontcharovd/.virtualenvs/r-reticulate/lib/python3.7/site-packages/tensorflow

python versions found:
 /home/gontcharovd/.virtualenvs/r-reticulate/bin/python
 /usr/bin/python3
 /home/gontcharovd/anaconda3/bin/python
 /home/gontcharovd/anaconda3/envs/tf-gpu/bin/python

dfalbel commented 5 years ago

I'll update the example as soon as possible. You could try the unet package available here: https://github.com/r-tensorflow/unet which has an updated implementation.

fenglinhuoshan123 commented 4 years ago

@gontcharovd @dfalbel Hi, I met with the same error on linux machine with my own neutral network;
but my codes could run correctly on Win10.

Any idea what could possilby be the problem? Tks！

Here is my error: `Error in py_call_impl(callable, dots$args, dots$keywords) : AttributeError: 'RGenerator' object has no attribute 'shape'

Detailed traceback: File "/home/jhlu/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site- packages/tensorflow_core/python/keras/engine/training.py", line 1297, in fit_generator steps_name='steps_per_epoch') File "/home/jhlu/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site- packages/tensorflow_core/python/keras/engine/training_generator.py", line 144, in model_iteration shuffle=shuffle) File "/home/jhlu/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site- packages/tensorflow_core/python/keras/engine/training_generator.py", line 477, in convert_to_generator_like num_samples = int(nest.flatten(data)[0].shape[0])`

rstudio / keras3