Closed faltinl closed 4 years ago
It seems that you have the correct version of the R package but, an older version of the python package installed. Can you try running:
tensorflow::install_tensorflow(version="2.0.0")
This will install the 2.0.0 python package.
Tnx for reply. I have tried that already and there were problems with R 3.6.2. So, this time, I downgraded to R 3.5.3 and started from scratch:
> devtools::install_github("rstudio/keras")
This installed keras 2.2.5.0, reticulate 1.14 tensorflow 2.0.0. The whole process was only interrupted by the prompt for installation of Miniconda and I did a restart R
before having run the next line:
> tensorflow::install_tensorflow(version="2.0.0")
Again, everything run flawlessly and I did a restart R
afterwards. Then, the usual sequence followed - unfortunately with a disappoiting result:
> library(reticulate) # 1.14
> use_condaenv("r-tensorflow", required = TRUE)
> library(tensorflow) # 2.0.0
> library(keras) # 2.2.5.0
> is_keras_available()
[1] FALSE
The usual tests led to results as follows:
> sessionInfo()
R version 3.5.3 (2019-03-11)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 18363)
Matrix products: default
locale:
[1] LC_COLLATE=German_Germany.1252 LC_CTYPE=German_Germany.1252
[3] LC_MONETARY=German_Germany.1252 LC_NUMERIC=C
[5] LC_TIME=German_Germany.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] keras_2.2.5.0 tensorflow_2.0.0 reticulate_1.14
loaded via a namespace (and not attached):
[1] Rcpp_1.0.3 rstudioapi_0.10 whisker_0.4 magrittr_1.5
[5] rappdirs_0.3.1 usethis_1.5.1 devtools_2.2.1 pkgload_1.0.2
[9] R6_2.4.1 rlang_0.4.2 fansi_0.4.1 tools_3.5.3
[13] pkgbuild_1.0.6 sessioninfo_1.1.1 cli_2.0.1 withr_2.1.2
[17] tfruns_1.4 ellipsis_0.3.0 remotes_2.1.0 assertthat_0.2.1
[21] digest_0.6.23 rprojroot_1.3-2 crayon_1.3.4 processx_3.4.1
[25] callr_3.4.0 base64enc_0.1-3 fs_1.3.1 ps_1.3.0
[29] zeallot_0.1.0 testthat_2.3.1 memoise_1.1.0 glue_1.3.1
[33] compiler_3.5.3 generics_0.0.2 desc_1.2.0 backports_1.1.5
[37] prettyunits_1.1.0 jsonlite_1.6
> reticulate::py_config()
python: C:/.../Local/conda/conda/envs/r-tensorflow/python.exe
libpython: C:/.../Local/conda/conda/envs/r-tensorflow/python36.dll
pythonhome: C:/.../Local/conda/conda/envs/r-tensorflow
version: 3.6.6 |Anaconda, Inc.| (default, Jun 28 2018, 11:27:44) [MSC v.1900 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/.../Local/conda/conda/envs/r-tensorflow/Lib/site-packages/numpy
numpy_version: 1.14.3
tensorflow: C:\...\Local\conda\conda\envs\R-TENS~1\lib\site-packages\tensorflow\__init__.p
NOTE: Python version was forced by use_python function
> tensorflow::tf_config()
TensorFlow v1.5.0 (C:\...\Local\conda\conda\envs\R-TENS~1\lib\site-packages\tensorflow\__init__.p)
Python v3.6 (C:/.../Local/conda/conda/envs/r-tensorflow/python.exe)
> implementation()
Module(tensorflow.keras)
> keras.backend.backend()
Error in keras.backend.backend(): could not find function "keras.backend.backend"
Thus, while tensorflow is working as it should, keras itself has problems. And since I didn't find any indications as to the reson for this behaviour whatsoever, I really need help. Thanks in advance...
is_keras_available()
is not the function you should use to check if your installation is working. It checks if the Keras python package is available, but it's not necessary in order to the Keras package to work because we are using the TensorFlow implementation.
The problem here is that for some reason you are still getting the TensorFlow v1.5 in your environment.
Can you try use_condaenv("r-reticulate", required = TRUE)
Note, the r-reticulate
instead of r-tensorflow
.
Tnx for suggestions. If after tensorflow::install_tensorflow(version="2.0.0")
I do a restart R
and then
library(reticulate)
use_condaenv("r-reticulate", required = TRUE) [Note: same test-results WITHOUT this line]
library(tensorflow)
library(keras)
I invariably get the following results for the tests:
> reticulate::py_config()
python: C:/Users/test/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython: C:/Users/test/AppData/Local/r-miniconda/envs/r-reticulate/python36.dll
pythonhome: C:/Users/test/AppData/Local/r-miniconda/envs/r-reticulate
version: 3.6.10 |Anaconda, Inc.| (default, Jan 7 2020, 15:18:16) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/test/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.18.1
tensorflow: C:\Users\test\AppData\Local\R-MINI~1\envs\R-RETI~1\lib\site-packages\tensorflow\__init__.p
NOTE: Python version was forced by use_python function
But then for tensorflow:
> tensorflow::tf_config()
Installation of TensorFlow not found.
Python environments searched for 'tensorflow' package:
C:\Users\test\AppData\Local\r-miniconda\envs\r-reticulate\python.exe
You can install TensorFlow using the install_tensorflow() function.
This changes into the message that tensorflow has been downgraded to v1.5 only in case of use_condaenv("r-tensorflow", required = TRUE)
. And finally an error message after the line
> library(keras)
Error: package or namespace load failed for ‘keras’:
.onLoad in loadNamespace() for 'keras' failed, details:
Call: py_module_import(module, convert = convert)
Error: ImportError: Traceback (most recent call last):
File "C:\Users\test\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\tensorflow_core\python\pywrap_tensorflow.py", line 58, in <module>
from tensorflow.python.pywrap_tensorflow_internal import *
File "C:\Users\test\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\tensorflow_core\python\pywrap_tensorflow_internal.py", line 28, in <module>
_pywrap_tensorflow_internal = swig_import_helper()
File "C:\Users\test\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\tensorflow_core\python\pywrap_tensorflow_internal.py", line 24, in swig_import_helper
_mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname, description)
File "C:\Users\test\AppData\Local\r-miniconda\envs\r-reticulate\lib\imp.py", line
...which I don't understand at all, but which also doesn't appear in case of use_condaenv("r-tensorflow", required = TRUE)
.
For your information: Since my last attempts I have removed completely R3.5.3, Miniconda, RStudio1.2.5033 (including a thorough clean-up of all associated directories) and reinstalled R3.6.1, and again RStudio1.2.5033 as well as Miniconda by means of tensorflow::install_tensorflow(version="2.0.0")
, but apparently to no avail - I'm really desperate by now.
For background: Ana-/Miniconda manages Python environments for you, where each environment can have completely different Python libraries.
So, please forget about the old r-tensorflow
, and just concentrate on r-reticulate
, which now has become the name of the default environment reticulate creates for you when/if it installs Miniconda.
Also, the reason to always use required=TRUE
is that if you don't, you won't know if it actually happened or not. (It can only happen if Python is initialized afresh, which normally only will happen if you restart the R session.)
Now, the easiest way to do some further testing would probably involve the command line. Normally,
conda activate r-reticulate
would "load" the respective environment. Then if you do
python
you should get a Python shell, and you should be able to try
import tensorflow as tf
Additionally - even before executing python
you could do
pip list
to see a list of installed libraries. Can you let us know what that yields?
Hi, tnx a lot for the very useful explanation. import tensorflow as tf
got me a lengthy error message. Since I've never used python / python shell: how do I copy this message? Copy/Paste obviously doesn't work, sry.
In the meantime I include a screen shot.
Here is the pip list, again as a PNG...
So tensorflow isn't there... can you do - right there on the command line, after
conda activate r-reticulate
:
pip install tensorflow
and if that fails, paste the error message, otherwise try importing again?
OK, seems to have worked, tensorflow 2.1.0, though.
yeah, that's the current releasè since last week
if you need 2.0, do
pip uninstall tensorflow
pip install tensorflow==2.0
Well, in the meantime I got this with tf2.1.0:
Don't really understand why it shouldn't have worked - the installed components are all well listed:
My last two screen shots have perhaps been taken a bit too early. I had another go at import tensorflow as tf
and got an error message very similar (except for a few, but definitely not all indicated line numbers) to the first one:
Strange enough: I did another pip list
which shows: tensorflow 2.1.0
is definitely there (only the area with the tf-components is shown):
So - why then is it not imported?
Observed error traces look similar to error report #22794, "ImportError: DLL load failed: The specified module could not be found" - with even several code line numbers coninciding. Case #22512 seems to go into a remotely similar direction.
am also facing same issues .. Kindly help..
Whatever I do (I have changed to Anaconda3 w/ Py3.6.10, in the meantime), I end up with
> reticulate::py_config()
python: C:/Anaconda/envs/r-reticulate/python.exe
libpython: C:/Anaconda/envs/r-reticulate/python36.dll
pythonhome: C:/Anaconda/envs/r-reticulate
version: 3.6.10 |Anaconda, Inc.| (default, Jan 7 2020, 15:18:16) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Anaconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.18.1
tensorflow: C:\Anaconda\envs\R-RETI~1\lib\site-packages\tensorflow\__init__.p
NOTE: Python version was forced by use_python function
and then - although install_tensorflow()
was indicated to have worked correctly every times I applied it -
> tensorflow::tf_config()
Installation of TensorFlow not found.
Python environments searched for 'tensorflow' package:
C:\Anaconda\envs\r-reticulate\python.exe
You can install TensorFlow using the install_tensorflow() function.
What can I do to get Tensorflow in working condition? Not to forget that I still need to get Keras working too...
Please help.
Not to forget that I still need to get Keras working too...
On the upside, Keras is part of TF, there'll be nothing to do there ;-)
One of the issues you're linking to hints to the problem possibly being linked to Windows GPU binaries now requiring "Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019".
To check if this is the issue, can you do
conda activate r-reticulate
pip uninstall tensorflow
pip install tensorflow-cpu
For background, since 2.1 the default binary is the GPU-enabled one; the CPU-only one can be installed separately.
Then when you test, please make sure you're in the right environment (r-reticulate
) always.
Tnx for reply & suggestions which I understand quite well. However, since I did so many experiments (as I would rather call them by now), I had already removed Tensorflow 2.1.0 in the meantime. So the problems observed apparently have nothing to do with either 2.1.0 nor 2.0.0. And rightly so: after installing tensorflow-cpu as indicated above, an error message like the last one appeared again. So essentially, nothing has changed, I have tensorflow 2.1.0 again in my pip list
; RStudio package list, however, still shows 2.0.0. btw. Anyway, tensorflow::tf_config()
still shows Installation of TensorFlow not found
.
At the moment I suspect my problems have to do with the path leading to tensorflow. I found an issue dealing with that, where quite similar problems like mine disappeard after doing the tensorflow-installation with
install_tensorflow(method="conda",conda=[path])
but: how do I find the [path] when even the programme itself doesn't find it? Simply doing install_tensorflow(method="conda")
doesn't help anyway, I have tried that one already...
PS: Oh, yes - and the "Microsoft Visual C++ Redistributable for Visual Studio 2015, 2017 and 2019" I have already installed, too, in the meantime.
I had already removed Tensorflow 2.1.0 in the meantime. So the problems observed apparently have nothing to do with either 2.1.0 nor 2.0.0. And rightly so: after installing tensorflow-cpu as indicated above, an error message like the last one appeared again. So essentially, nothing has changed, I have tensorflow 2.1.0 again in my pip list;
I do not follow ... You removed 2.1 yet pip says you have it?
At this point, I can only suggest the same as above
conda activate r-reticulate
pip uninstall tensorflow
pip install tensorflow-cpu
python
> import tensorflow as tf
Alternatively:
conda activate r-reticulate
pip uninstall tensorflow
pip install tensorflow=2.0
python
> import tensorflow as tf
We first need to get it working from Python, and then see about R.
https://github.com/tensorflow/tensorflow/issues/35749 https://github.com/tensorflow/tensorflow/issues/35618
Yes, as I said: After our attempts from 4 days ago with tensorflow 2.1.0 I deinstalled that, then repeated the process with 2.0.0, a second time with 2.1.0 and a second time with 2.0.0. And from this last attempt obviously remained one tensorflow 2.1.0, together with the last installation of tensorflow 2.0.0, see pip list from just a few mins after I received your last post:
Since there was an apparently already complete set of tensorflow 2.0.0 I decided to follow your alternative, uninstalled with pip uninstall tensorflow
the default version, i.e. tensorflow 2.1.0, and installed pip install tensorflow==2.0
. This produced a long trace saying for every component Requirement already satisfied
(everything was there already, as expected), and so I tried import tensorflow as tf
under python
. This however, didn't work, see pic:
Apparently, something went wrong with this "partial" uninstall/install actions or so, don't know. I therefore uninstalled tensorflow 2.0.0 (i.e. definitely both versions were then absent), and finally did pip install tensorflow==2.0
again, successfully as it said. Changing to python
shell again and trying to import tensorflow as tf
again produced the well known long error trace from a few posts above.
From within RStudio nothing has changed either, as the command sequence
library(reticulate)
use_condaenv("r-reticulate", required = TRUE)
library(tensorflow)
again gives
> tensorflow::tf_config()
Installation of TensorFlow not found.
...
In the meantime I changed from R3.6.1 (when installing R-packages, I got several warnings that they have been built under R3.6.2) to R3.6.2 and from Anaconda3 to Miniconda3 - to no avail: Not only did I get the same message from tensorflow::tf_config()
as above, but also trying to import tensorflow as tf
(v2.0.0) under python
led to the same lengthy error message shown further above.
Against much earlier attemps the situation has changed insofar, as I now keep getting identical error messages consistently (i.e. since about 10 days intensive trials). This indicates, that there must be a rather basic problem, common to all the undertaken attempts, behind this installation problem. As I have not the slightest idea what that could be, I'm asking desperately for help.
I don't like saying this, but given that your issue looks so similar to the ones I linked in my comment above (and I think when you say, you get one error message again and again, it's the DLL not found one, right?), while these two issues were closed in TF either with downgrade to 2.0 or installing that Visual Studio thing ... I think it makes most sense for you to create an issue in the TF repo directly.
They might have an idea which DLL is not found, or how to find out...
Sorry for not having better advice in this situation...
If you create that issue in TF, please link it here so we can follow.
Tnx. The issue was forwarded to Tensorflow 2.0.0 cpu, import error: DLL load failed #36138.
Ok, let's hope for the best :-)
BTW you may want to edit the issue and remove anything R/RStudio-related, as it is not relevant and could only be distracting :-)
I thought so too, but looking at the actual install guides makes me nervous: if the so called 'simple' and 'quick' installation via RStudio should really not be possible (it looks as if nobody seems to be interested in that any more), I would have to forget my running ML projects for which I have worked the past 3 years for quite a long time. To learn Python and the handling of several auxiliary 'environments' etc. just for being able to bring tensorflow back to life on my computer is no fun for people like me who are interested in applications and not in IT itself...
I should have been more clear - what reticulate
/ tensorflow
do is nothing than install TF for you; they do exactly the same thing you ended up doing with Python on the command line.
But if an error occurs that is directly attributable to the TF installation per se (not to R not managing to find it, say), all that is needed to diagnose is information about the Python installation.
Even if the problem has nothing to do with R, we always try to help. But in some cases, like yours, it may make sense to file an issue in the original repository. Especially since your issue is so similar to ones that were closed with solutions none of which worked for you.
(BTW, added a comment to your issue in that respect.)
In connection with the new version of reticulate v1.14 and after quite a few problems to get Keras available, I encountered a rather serious problem. In an existing program written under
keras v2.2.0.9000
,reticulate v1.10
andtensorflow v1.9
and since then run successfully many times, the commandnow run under
reticulate 1.14
,tensorflow 2.0.0
andkeras 2.2.5.0
, produce the following error message:Similarly, the command
produces the error message
So - which method would then be applicable for compilation? Consulting the
keras 2.2.5.0
index w.r.t. commandcompile
shows thatcompile
as such doesn't exist as entry in the index any more, but onlyHowever, under this symbolic headline I find the same description as earlier under keyword
compile(...)
and I cannnot detect anything wrong within the command used as above. Correspondingly the situation forfit()
.Moreover, the function
to_categorical(y, num_classes = NULL, ...)
now seems to require specification ofnum_classes
explicitely, since without this specification (as it was used up to now), a mysterious error messageappears. And, really, the command
tensorflow::tf_config()
shows, much to my surprise,Note that in my case actually
tensorflow v2.0.0
has been installed; the recommended command to installtensorflow v1.9
(sic!) at this point sounds like a dire reminiscence to my (lost) previous era with Keras. And there is no way back: an attempt to reinstalltensorflow 1.9
automatically leads to reinstallation oftensorflow 2.0.0
, sincetensorflow 1.9
is not available forR 3.6.2
. Catch 362, so to speak...I am really at a loss how to proceed. Any useful hint would be highly appreciated.
Tnx in advance, Leo Faltin