Errors when running ee_Initialize() and ee_check()

AcademicusChamaeleon commented 2 years ago

rgee version: 1.1.0
R version: 4.1.2
Operating System: Windows 10

> library(reticulate)
> py_config()
python:         C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython:      C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/python36.dll
pythonhome:     C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate
version:        3.6.13 (default, Sep 23 2021, 07:38:49) [MSC v.1916 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version:  1.19.5

Description

It had errors when I ran ee_Initialize() and ee_check()

What I Did

> ee_install()

---------------------------------------------------------------------------------------------------------------------- Python configuration used to create rgee --
python:         C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/python.exe
libpython:      C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/python36.dll
pythonhome:     C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate
version:        3.6.13 (default, Sep 23 2021, 07:38:49) [MSC v.1916 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:/Users/yunti/AppData/Local/r-miniconda/envs/r-reticulate/Lib/site-packages/numpy
numpy_version:  1.19.5
------------------------------------------------------------------------------------------------------------------------------------------------------------------ 
1. Removing the previous Python Environment (rgee), if it exists ...

Remove all packages in environment C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee:

## Package Plan ##

  environment location: C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee

The following packages will be REMOVED:

  aiohttp-3.7.4.post0-py310he2412df_1
  async-timeout-3.0.1-py_1000
  attrs-21.2.0-pyhd8ed1ab_0
  bzip2-1.0.8-h8ffe710_4
  ca-certificates-2021.10.8-h5b45459_0
  cachetools-4.2.4-pyhd8ed1ab_0
  chardet-4.0.0-py310h5588dad_2
  earthengine-api-0.1.186-py_0
  google-api-python-client-1.7.12-pyh8c360ce_0
  google-auth-1.25.0-pyh44b312d_0
  google-auth-httplib2-0.1.0-pyhd8ed1ab_0
  httplib2-0.20.2-pyhd8ed1ab_1
  idna-3.1-pyhd3deb0d_0
  intel-openmp-2021.4.0-h57928b3_3556
  libblas-3.9.0-12_win64_mkl
  libcblas-3.9.0-12_win64_mkl
  libffi-3.4.2-h8ffe710_5
  liblapack-3.9.0-12_win64_mkl
  libzlib-1.2.11-h8ffe710_1013
  mkl-2021.4.0-h0e2418a_729
  multidict-5.2.0-py310he2412df_1
  numpy-1.21.4-py310h5ebf175_0
  openssl-3.0.0-h8ffe710_2
  pip-21.3.1-pyhd8ed1ab_0
  pyasn1-0.4.8-py_0
  pyasn1-modules-0.2.7-py_0
  pyparsing-3.0.6-pyhd8ed1ab_0
  python-3.10.0-hcf16a7b_2_cpython
  python_abi-3.10-2_cp310
  rsa-4.7.2-pyh44b312d_0
  setuptools-59.1.1-py310h5588dad_0
  simplejson-3.17.5-py310he2412df_1
  six-1.16.0-pyh6c4a22f_0
  sqlite-3.36.0-h8ffe710_2
  tbb-2021.4.0-h2d74725_1
  tk-8.6.11-h8ffe710_1
  typing-extensions-3.10.0.2-hd8ed1ab_0
  typing_extensions-3.10.0.2-pyha770c72_0
  tzdata-2021e-he74cb21_0
  ucrt-10.0.20348.0-h57928b3_0
  uritemplate-3.0.1-py_0
  vc-14.2-hb210afc_5
  vs2015_runtime-14.29.30037-h902a5da_5
  wheel-0.37.0-pyhd8ed1ab_1
  xz-5.2.5-h62dcd97_1
  yarl-1.7.2-py310he2412df_1

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

2. Creating a Python Environment (rgee)
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee

  added / updated specs:
    - python

The following NEW packages will be INSTALLED:

  bzip2              conda-forge/win-64::bzip2-1.0.8-h8ffe710_4
  ca-certificates    conda-forge/win-64::ca-certificates-2021.10.8-h5b45459_0
  libffi             conda-forge/win-64::libffi-3.4.2-h8ffe710_5
  libzlib            conda-forge/win-64::libzlib-1.2.11-h8ffe710_1013
  openssl            conda-forge/win-64::openssl-3.0.0-h8ffe710_2
  pip                conda-forge/noarch::pip-21.3.1-pyhd8ed1ab_0
  python             conda-forge/win-64::python-3.10.0-hcf16a7b_2_cpython
  python_abi         conda-forge/win-64::python_abi-3.10-2_cp310
  setuptools         conda-forge/win-64::setuptools-59.1.1-py310h5588dad_0
  sqlite             conda-forge/win-64::sqlite-3.36.0-h8ffe710_2
  tk                 conda-forge/win-64::tk-8.6.11-h8ffe710_1
  tzdata             conda-forge/noarch::tzdata-2021e-he74cb21_0
  ucrt               conda-forge/win-64::ucrt-10.0.20348.0-h57928b3_0
  vc                 conda-forge/win-64::vc-14.2-hb210afc_5
  vs2015_runtime     conda-forge/win-64::vs2015_runtime-14.29.30037-h902a5da_5
  wheel              conda-forge/noarch::wheel-0.37.0-pyhd8ed1ab_1
  xz                 conda-forge/win-64::xz-5.2.5-h62dcd97_1

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
#
# To activate this environment, use
#
#     $ conda activate rgee
#
# To deactivate an active environment, use
#
#     $ conda deactivate

rgee::ee_install want to store the environment variables: EARTHENGINE_PYTHON 
and EARTHENGINE_ENV in your .Renviron file to use the Python path:
C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee\python.exe in future sessions.
Would you like to continues? [Y/n]:y

3. The Environment Variables: 'EARTHENGINE_PYTHON=C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee\python.exe' and 'EARTHENGINE_ENV=rgee'
were saved in the .Renviron (C:/Users/yunti/Documents/.Renviron) file. Remember that you
could remove EARTHENGINE_PYTHON and EARTHENGINE_ENV using rgee::ee_clean_pyenv().

4. Installing the earthengine-api. Running: 
reticulate::py_install(packages = 'earthengine-api', envname = 'C:/Users/yunti/AppData/Local/r-miniconda/envs/rgee')

Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: C:\Users\yunti\AppData\Local\r-miniconda\envs\rgee

  added / updated specs:
    - earthengine-api
    - numpy

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    aiohttp-3.8.1              |  py310he2412df_0         563 KB  conda-forge
    aiosignal-1.2.0            |     pyhd8ed1ab_0          12 KB  conda-forge
    async-timeout-4.0.1        |     pyhd8ed1ab_0           9 KB  conda-forge
    charset-normalizer-2.0.7   |     pyhd8ed1ab_0          33 KB  conda-forge
    frozenlist-1.2.0           |  py310he2412df_1          80 KB  conda-forge
    simplejson-3.17.6          |  py310he2412df_0         105 KB  conda-forge
    typing-extensions-4.0.0    |       hd8ed1ab_0           8 KB  conda-forge
    typing_extensions-4.0.0    |     pyha770c72_0          26 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         836 KB

The following NEW packages will be INSTALLED:

  aiohttp            conda-forge/win-64::aiohttp-3.8.1-py310he2412df_0
  aiosignal          conda-forge/noarch::aiosignal-1.2.0-pyhd8ed1ab_0
  async-timeout      conda-forge/noarch::async-timeout-4.0.1-pyhd8ed1ab_0
  attrs              conda-forge/noarch::attrs-21.2.0-pyhd8ed1ab_0
  cachetools         conda-forge/noarch::cachetools-4.2.4-pyhd8ed1ab_0
  charset-normalizer conda-forge/noarch::charset-normalizer-2.0.7-pyhd8ed1ab_0
  earthengine-api    conda-forge/noarch::earthengine-api-0.1.186-py_0
  frozenlist         conda-forge/win-64::frozenlist-1.2.0-py310he2412df_1
  google-api-python~ conda-forge/noarch::google-api-python-client-1.7.12-pyh8c360ce_0
  google-auth        conda-forge/noarch::google-auth-1.25.0-pyh44b312d_0
  google-auth-httpl~ conda-forge/noarch::google-auth-httplib2-0.1.0-pyhd8ed1ab_0
  httplib2           conda-forge/noarch::httplib2-0.20.2-pyhd8ed1ab_1
  idna               conda-forge/noarch::idna-3.1-pyhd3deb0d_0
  intel-openmp       conda-forge/win-64::intel-openmp-2021.4.0-h57928b3_3556
  libblas            conda-forge/win-64::libblas-3.9.0-12_win64_mkl
  libcblas           conda-forge/win-64::libcblas-3.9.0-12_win64_mkl
  liblapack          conda-forge/win-64::liblapack-3.9.0-12_win64_mkl
  mkl                conda-forge/win-64::mkl-2021.4.0-h0e2418a_729
  multidict          conda-forge/win-64::multidict-5.2.0-py310he2412df_1
  numpy              conda-forge/win-64::numpy-1.21.4-py310h5ebf175_0
  pyasn1             conda-forge/noarch::pyasn1-0.4.8-py_0
  pyasn1-modules     conda-forge/noarch::pyasn1-modules-0.2.7-py_0
  pyparsing          conda-forge/noarch::pyparsing-3.0.6-pyhd8ed1ab_0
  rsa                conda-forge/noarch::rsa-4.7.2-pyh44b312d_0
  simplejson         conda-forge/win-64::simplejson-3.17.6-py310he2412df_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  tbb                conda-forge/win-64::tbb-2021.4.0-h2d74725_1
  typing-extensions  conda-forge/noarch::typing-extensions-4.0.0-hd8ed1ab_0
  typing_extensions  conda-forge/noarch::typing_extensions-4.0.0-pyha770c72_0
  uritemplate        conda-forge/noarch::uritemplate-3.0.1-py_0
  yarl               conda-forge/win-64::yarl-1.7.2-py310he2412df_1

Downloading and Extracting Packages
typing-extensions-4. | 8 KB      | ########## | 100% 
charset-normalizer-2 | 33 KB     | ########## | 100% 
aiosignal-1.2.0      | 12 KB     | ########## | 100% 
typing_extensions-4. | 26 KB     | ########## | 100% 
async-timeout-4.0.1  | 9 KB      | ########## | 100% 
aiohttp-3.8.1        | 563 KB    | ########## | 100% 
frozenlist-1.2.0     | 80 KB     | ########## | 100% 
simplejson-3.17.6    | 105 KB    | ########## | 100% 
Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done

Well done! rgee was successfully set up in your system.
You need restart R to see changes. After doing that, we recommend
run ee_check() to perform a full check of all non-R rgee dependencies.
Do you want restart your R session? 

1: yes
2: no

Selection: 1

Restarting R session...

> ee_Initialize()
Error in ee_connect_to_py(path = ee_current_version, n = 5) : 
  The current Python PATH: C:/Users/yunti/AppData/Local/r-miniconda/envs/rgee/python.exe
does not have the Python package "earthengine-api" installed. Are you restarted/terminated.
your R session after install miniconda or run ee_install()?
If no do it, please try again:

> ee_check()
(*)  Python version
√ [Ok] C:/Users/yunti/AppData/Local/r-miniconda/envs/rgee/python.exe v3.1
Error in ee_check_python(quiet = quiet) : rgee needs Python 3.5 >=

jordi-1 commented 2 years ago

I got exactly the same problem. Wondering if it's not related to code unable to differentiate between Python 3.1 and 3.10 recently released ?

csaybar commented 2 years ago

Hi can restart your R session and then run:

rgee_environment_dir = "C:\\Users\\yunti\\AppData\\Local\\r-miniconda\\envs\\rgee\\"
Sys.setenv(RETICULATE_PYTHON = rgee_environment_dir)

reticulate::py_run_string("import ssl")

and show us the result?

csaybar commented 2 years ago

Thanks to @ryali93, we found the bug. It is a really specific issue, it only happens if you are using:

Windows
Python 3.10 (installed by default by miniconda).
are calling python using reticulate.
loads the ssl python package.

The bug in reticulate occur exactly here: https://github.com/rstudio/reticulate/blob/dd6d6e8139dcfc4ce19e6b7150406a6f4e1af747/inst/python/rpytools/loader.py#L39

We will try to investigate deeper the error. But for now, a workaround would be to use python 3.9 instead of 3.10. I highly recommend you follow the @ricds tutorial.

You will find the code to install here: https://github.com/ricds/DL_RS_GEE/blob/main/rgee_install_packages.R and a video tutorial here: https://www.youtube.com/watch?v=1-k6wNL2hlo

patchcervan commented 2 years ago

I don't know if this has anything to do with anything, but this line in ee_check_python() returns 3.1 instead of 3.10: py_version <- as.numeric(py_discover_config()[["version"]])

P.S. thanks for the handy package!

csaybar commented 2 years ago

Hi @patchcervan Yes!!, unfortunately, this happens in reticulate and not rgee :( Please try to install manually Python 3.9 rather than 3.10. rgee should work again then.

patchcervan commented 2 years ago

I see... python 3.9 does work for now, thanks!

csaybar commented 2 years ago

Hi, the problem has been solved!. Please run as follow to fix!

remotes::install_github("r-spatial/rgee")
library(rgee)
ee_install()

r-spatial / rgee

Errors when running ee_Initialize() and ee_check() #196

Description

What I Did