problem when update to V0.5.12

zoumuda commented 3 years ago

Hi, I installed ADCME v0.5.07 successfully on windows, however when I update ADCME to v0.5.12, it doesn't work. It said "you did not install tensorflow in the python version being used by PyCall".Could you help me? Thanks.

kailaix commented 3 years ago

Hi, we made a breaking change since v0.5.10 to improve dependency stability. You can try remove the directory ~/.julia/conda (in windows, it should be something like C:\Users\<USERNAME>\.julia\conda\ and rebuild ADCME

using Pkg
Pkg.build("ADCME")

zoumuda commented 3 years ago

Hi, we made a breaking change since v0.5.10 to improve dependency stability. You can try remove the directory ~/.julia/conda (in windows, it should be something like C:\Users\<USERNAME>\.julia\conda\ and rebuild ADCME
using Pkg
Pkg.build("ADCME")

Thanks. But It still does not work. May I know the meaning of "remove the directory '~/.julia/conda'"? Does it mean delete the folder named "conda" in the directory '~/.julia/'. It does not work so far. The following is the error.

" ERROR: LoadError: PyError (PyImport_ImportModule │ │ The Python package tensorflow could not be found by pyimport. Usually this means
│ that you did not install tensorflow in the Python version being used by PyCall. "

kailaix commented 3 years ago

I realize you are using Windows. There is an extra step for Windows: could you add the corresponding path to your environment variable? See the blue box Info section https://kailaix.github.io/ADCME.jl/dev/#Installation. Make sure you use your own username in the path. Then rebuild ADCME

using Pkg
Pkg.build("ADCME")

This extra step may be unnecessary in a future version once we figure out a way to resolve all the dependencies issues on Windows. Let me know if it doesn't work.

BTW, could you also paste the full error message here?

zoumuda commented 3 years ago

I realize you are using Windows. There is an extra step for Windows: could you add the corresponding path to your environment variable? See the blue box Info section https://kailaix.github.io/ADCME.jl/dev/#Installation. Make sure you use your own username in the path. Then rebuild ADCME
using Pkg
Pkg.build("ADCME")
This extra step may be unnecessary in a future version once we figure out a way to resolve all the dependencies issues on Windows. Let me know if it doesn't work.

BTW, could you also paste the full error message here?

Thank you for your concern. I added the three path to my evironment as you mentioned. But it still does not work. I past the error message. Hope it can be solved.

julia> using Pkg

julia> Pkg.build("ADCME") Building Conda ─→ C:\Users\zhao\.julia\packages\Conda\3rPhK\deps\build.log Building PyCall → C:\Users\zhao\.julia\packages\PyCall\zqDXB\deps\build.log Building CMake ─→ C:\Users\zhao\.julia\packages\CMake\ULbyn\deps\build.log Building HDF5 ──→ C:\Users\zhao\.julia\packages\HDF5\T1b9x\deps\build.log Building FFTW ──→ C:\Users\zhao\.julia\packages\FFTW\DMUbN\deps\build.log Building ADCME ─→ C:\Users\zhao\.julia\packages\ADCME\PJIHk\deps\build.log ┌ Error: Error building ADCME: │ ┌ Warning: Pkg.installed() is deprecated │ └ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531 │ ┌ Warning: Pkg.installed() is deprecated │ └ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531 │ [ Info: Your Julia version is 1.4.1, current ADCME version is 0.5.12, ADCME dependencies installation path: C:\Users\zhao.julia\adcme │ [ Info: --------------- (1/6) Install Tensorflow Dependencies --------------- │ [ Info: ADCME dependencies have already been installed │ [ Info: --------------- (2/6) Check Python Version --------------- │ ┌ Warning: Pkg.installed() is deprecated │ └ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531 │ Building Conda ─→ C:\Users\zhao\.julia\packages\Conda\3rPhK\deps\build.log │ Building PyCall → C:\Users\zhao\.julia\packages\PyCall\zqDXB\deps\build.log │ ┌ Info: PyCall Python version: C:\Users\zhao.julia\adcme\python.exe │ └ Conda Python version: C:\Users\zhao.julia\adcme\python.exe │ [ Info: --------------- (3/6) Looking for TensorFlow Dynamic Libraries --------------- │ ERROR: LoadError: PyError (PyImport_ImportModule │ │ The Python package tensorflow could not be found by pyimport. Usually this means │ that you did not install tensorflow in the Python version being used by PyCall. │ │ PyCall is currently configured to use the Python version at: │ │ C:\Users\zhao.julia\adcme\python.exe │ │ and you should use whatever mechanism you usually use (apt-get, pip, conda, │ etcetera) to install the Python package containing the tensorflow module. │ │ One alternative is to re-configure PyCall to use a different Python │ version on your system: set ENV["PYTHON"] to the path/name of the python │ executable you want to use, run Pkg.build("PyCall"), and re-launch Julia. │ │ Another alternative is to configure PyCall to use a Julia-specific Python │ distribution via the Conda.jl package (which installs a private Anaconda │ Python distribution), which has the advantage that packages can be installed │ and kept up-to-date via Julia. As explained in the PyCall documentation, │ set ENV["PYTHON"]="", run Pkg.build("PyCall"), and re-launch Julia. Then, │ To install the tensorflow module, you can use pyimport_conda("tensorflow", PKG), │ where PKG is the Anaconda package the contains the module tensorflow, │ or alternatively you can use the Conda package directly (via │ using Conda followed by Conda.add etcetera). │ │ ) <class 'ImportError'> │ ImportError('\n\nIMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!\n\nImporting the numpy C-extensions failed. This error can happen for\nmany reasons, often due to issues with your setup or how NumPy was\ninstalled.\n\nWe have compiled some common reasons and troubleshooting tips at:\n\n https://numpy.org/devdocs/user/troubleshooting-importerror.html\n\nPlease note and check the following:\n\n The Python version is: Python3.7 from "C:\Users\zhao\AppData\Local\Programs\Julia\Julia-1.4.1\bin\julia.exe"\n The NumPy version is: "1.19.1"\n\nand make sure that they are the versions you expect.\nPlease carefully study the documentation linked above for further help.\n\nOriginal error was: DLL load failed: The specified module could not be found.\n') │ File "C:\Users\zhao.julia\adcme\lib\site-packages\tensorflow__init.py", line 99, in │ from tensorflow_core import * │ File "C:\Users\zhao.julia\adcme\lib\site-packages\tensorflow_core__init.py", line 34, in │ from tensorflow.python import pywrap_tensorflow # pylint: disable=unused-import │ File "C:\Users\zhao.julia\adcme\lib\site-packages\tensorflow__init.py", line 50, in getattr__ │ module = self._load() │ File "C:\Users\zhao.julia\adcme\lib\site-packages\tensorflow\init__.py", line 44, in _load │ module = _importlib.import_module(self.name) │ File "C:\Users\zhao.julia\adcme\lib\importlib__init__.py", line 127, in import_module │ return _bootstrap._gcd_import(name[level:], package, level) │ File "C:\Users\zhao.julia\adcme\lib\site-packages\tensorflow_core\python\init.py", line 47, in │ import numpy as np │ File "C:\Users\zhao.julia\adcme\lib\site-packages\numpy\init.py", line 140, in │ from . import core │ File "C:\Users\zhao.julia\adcme\lib\site-packages\numpy\core\init__.py", line 48, in │ raise ImportError(msg) │ │ Stacktrace: │ [1] pyimport(::String) at C:\Users\zhao.julia\packages\PyCall\zqDXB\src\PyCall.jl:536 │ [2] top-level scope at C:\Users\zhao.julia\packages\ADCME\PJIHk\deps\build.jl:69 │ [3] include(::String) at .\client.jl:439 │ [4] top-level scope at none:5 │ in expression starting at C:\Users\zhao.julia\packages\ADCME\PJIHk\deps\build.jl:69 └ @ Pkg.Operations D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Operations.jl:892

kailaix commented 3 years ago

Hi, I tried to reproduce your error but was not able to. It seems that your old installation is causing problems. You need to remove the old ~/.julia/adcme directory. For example, on my computer, I have a directory C:\Users\kaila\.julia\adcme (my user name is kaila, and you need to replace it with yours, zhao). You should see a message like

  Building ADCME ────→ `C:\Users\kaila\.julia\packages\ADCME\PJIHk\deps\build.log`
┌ Warning: Pkg.installed() is deprecated
└ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531
┌ Warning: Pkg.installed() is deprecated
└ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531
[ Info: Your Julia version is 1.4.1, current ADCME version is 0.5.12, ADCME dependencies installation path: C:\Users\kaila\.julia\adcme
[ Info:  --------------- (1/6) Install Tensorflow Dependencies  ---------------
[ Info: Installing miniconda...
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 4.8.5

Please update conda by running

    $ conda update -n base -c defaults conda

Downloading and Extracting Packages
cachetools-4.1.1     | 12 KB     | ############################################################################ | 100%
blas-1.0             | 6 KB      | ############################################################################ | 100%
tensorflow-1.15.0    | 4 KB      | ############################################################################ | 100%
markdown-3.2.2       | 61 KB     | ############################################################################ | 100%
wrapt-1.12.1         | 45 KB     | ############################################################################ | 100%
conda-4.8.4          | 3.1 MB    | ############################################################################ | 100%
tensorflow-base-1.15 | 36.2 MB   | ############################################################################ | 100%
requests-oauthlib-1. | 21 KB     | ############################################################################ | 100%
openssl-1.1.1g       | 4.8 MB    | ############################################################################ | 100%
libmklml-2019.0.5    | 17.4 MB   | ############################################################################ | 100%
tensorboard-2.3.0    | 6.2 MB    | ############################################################################ | 100%
mkl-service-2.3.0    | 52 KB     | ############################################################################ | 100%
liblapack-3.8.0      | 3.5 MB    | ############################################################################ | 100%
tensorflow-estimator | 271 KB    | ############################################################################ | 100%
...

That means ADCME does install the correct tensorflow backend for you.

zoumuda commented 3 years ago

Hi, I tried to reproduce your error but was not able to. It seems that your old installation is causing problems. You need to remove the old ~/.julia/adcme directory. For example, on my computer, I have a directory C:\Users\kaila\.julia\adcme (my user name is kaila, and you need to replace it with yours, zhao). You should see a message like

  Building ADCME ────→ `C:\Users\kaila\.julia\packages\ADCME\PJIHk\deps\build.log`
┌ Warning: Pkg.installed() is deprecated
└ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531
┌ Warning: Pkg.installed() is deprecated
└ @ Pkg D:\buildbot\worker\package_win64\build\usr\share\julia\stdlib\v1.4\Pkg\src\Pkg.jl:531
[ Info: Your Julia version is 1.4.1, current ADCME version is 0.5.12, ADCME dependencies installation path: C:\Users\kaila\.julia\adcme
[ Info:  --------------- (1/6) Install Tensorflow Dependencies  ---------------
[ Info: Installing miniconda...
Collecting package metadata (repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 4.8.5

Please update conda by running

    $ conda update -n base -c defaults conda

Downloading and Extracting Packages
cachetools-4.1.1     | 12 KB     | ############################################################################ | 100%
blas-1.0             | 6 KB      | ############################################################################ | 100%
tensorflow-1.15.0    | 4 KB      | ############################################################################ | 100%
markdown-3.2.2       | 61 KB     | ############################################################################ | 100%
wrapt-1.12.1         | 45 KB     | ############################################################################ | 100%
conda-4.8.4          | 3.1 MB    | ############################################################################ | 100%
tensorflow-base-1.15 | 36.2 MB   | ############################################################################ | 100%
requests-oauthlib-1. | 21 KB     | ############################################################################ | 100%
openssl-1.1.1g       | 4.8 MB    | ############################################################################ | 100%
libmklml-2019.0.5    | 17.4 MB   | ############################################################################ | 100%
tensorboard-2.3.0    | 6.2 MB    | ############################################################################ | 100%
mkl-service-2.3.0    | 52 KB     | ############################################################################ | 100%
liblapack-3.8.0      | 3.5 MB    | ############################################################################ | 100%
tensorflow-estimator | 271 KB    | ############################################################################ | 100%
...

That means ADCME does install the correct tensorflow backend for you.

Thank you for your time and great effort. Finally, it works. Hope everything goes well for you.

kailaix / ADCME.jl

problem when update to V0.5.12 #58