Set RETICULATE_PYTHON env var

yuvipanda commented 2 years ago

Based on https://tensorflow.rstudio.com/installation/custom/

For https://2i2c.freshdesk.com/a/tickets/65

github-actions[bot] commented 2 years ago

:point_left: Test this PR on Binder

damianavila commented 2 years ago

Makes sense accordingly to the docs, although in the test image I successfully tried in the past few days, the env variable was not actually needed (it seemed TensorFlow and Keras from the R side was able to auto-discover the corresponding Python packages). Wondering if the latest changes introduced on the R side with the latest merged PRs actually changed somehow the requirement for that environment variable 🤔

yuvipanda commented 2 years ago

@damianavila were you able to get some sample code to run? Importing the libraries works, but trying to actually do anything fails with these errors.

damianavila commented 2 years ago

Yes, I was able to run the following snippets:

library(tensorflow)
tf$constant("Hellow Tensorflow")

And with Keras the following lines worked as expected...

library(keras)
num_words <- 10
imdb <- dataset_imdb(num_words = num_words)

I tried again from the 2i2c staging and still works with a more complex Keras example...

and I do not see RETICULATE_PYTHON in the env variables...

yuvipanda commented 2 years ago

@damianavila can you try:

That's the output Nathan was getting, and I am getting a different error about RETICULATE. When I solve that, I'm getting an even more bizzare error that looks openssl related...

Also, staging and prod are on the same image - I used staging to test how the image I built works.

damianavila commented 2 years ago

Also, staging and prod are on the same image - I used staging to test how the image I built works.

Just for clarification, my tests were done in the 2i2c cluster / staging hub because I do not have access to the UoT hubs yet.

damianavila commented 2 years ago

@yuvipanda, I can confirm Nathan's example is breaking at the time to load the data set on staging.pilot.2i2c.cloud as well.

and the kernel dies

All of this was using the Notebook because if I try to start R Studio...

(this is the first time I see this problem... in my previous attempts with the image I was loading RStudio just fine... btw, not sure if this is also happening or not in the UToronto hub although, from the reports in the tickets, it seems they could start RStudio without issues and I think you did it as well, @yuvipanda, so I am confused about what's happening 😉).

sgibson91 commented 2 years ago

@damianavila There have been a few CI redeploys from merges and I have checked that I haven't broken the deployer for CI/CD stuff against 2i2c staging - maybe double check that the change you made to pull the image hasn't been overwritten? Should be fine if you did it with the configurator though, right?

damianavila commented 2 years ago

Should be fine if you did it with the configurator though, right?

Yep, I did it with the Configurator, I do not think this is related to your changes, @sgibson91

damianavila commented 2 years ago

OK, I did another research iteration on this one because it is still bugging me :wink:

First, I am using the Toronto already deployed image on 2i2c staging (using the Configurator to point to quay.io/2i2c/utoronto-image:f3069be5964c) because I do not have access to Toronto yet as I referenced above.

When I tested examples on the Python side things worked as expected. I can even load the mnist dataset. This is the notebook I used: Tensorflow_and_Keras_test_from_Python.ipynb.

Then, when I try the R kernel inside the notebook app, RStudio, or a plain R terminal, the following code (which is being executed from the R side) fails as reported by Nathan:

library(keras)
mnist <- dataset_mnist()

Surprisingly (or not), running other pieces of TF or Keras code from the R side of things work as expected as shown in the following notebook: TF_and_Keras_test_from_R.ipynb

Finally, when I checked the reticulate configuration in the already deployed image, it is finding stuff properly (as the above notebook have shown):

reticulate::py_config()
keras:::keras_version() 

python:         /opt/conda/bin/python3
libpython:      /opt/conda/lib/libpython3.8.so
pythonhome:     /opt/conda:/opt/conda
version:        3.8.12 | packaged by conda-forge | (default, Oct 12 2021, 21:59:51)  [GCC 9.4.0]
numpy:          /opt/conda/lib/python3.8/site-packages/numpy
numpy_version:  1.22.1

python versions found: 
 /opt/conda/bin/python3
 /opt/conda/bin/python
[1] ‘2.7.0’

so I do not think we need to set up the RETICULATE_PYTHON environment variable this PR is promoting.

Still, we do not know what is actually happening with the mnist dataset when you try to interact with it from the R side of things and why it is failing and there are others reporting similar things without a definitive answer, ie. https://github.com/rstudio/rstudio/issues/6908.

yuvipanda commented 2 years ago

Thank you for investigating this more thoroughly, @damianavila! I'll close this, but what do we tell UofT folks now?

damianavila commented 2 years ago

what do we tell UofT folks now?

This is my latest attempt to explain the situation (essentially a summary of the things I found above): https://2i2c.freshdesk.com/a/tickets/65?note=80114002334

Btw, I went ahead and provided some information because silence is always bad, IMHO and experience.

GeorgianaElena commented 2 years ago

I've re-opened this PR because of some findings:

If we don't set RETICULATE_PYTHON env var, then RStudio users will be provided with instruction to install Miniconda, which will crate a new conda env with a different version of Python, that reticulate will load.
If miniconda isn't installed, then reticulate will load the right Python version (the one that @damianavila sees) and won't suggest installing it the next time. Things will work just fine now
But if miniconda is installed, we now ended up with multiple versions of Python, and reticulate is set to use the miniconda one To fix this, the user must start a new RStudio session and tell reticulate to use the correct version of Python (/opt/conda/bin/Python) before running anything that will cause reticulate to use the default (r-miniconda/..) it has set This will make RStudio find the correct pkgs.

In case we want users to always type reticulate::use_python("/opt/conda/bin/Python") in every RStudio new session, then I believe we should set this env var and set the default Python for everyone? What do folks think?

damianavila commented 2 years ago

Just for completeness, I was not able to see the miniconda installation prompt in the 2i2c staging hub because, at some time I guess I said NO, and that was permanently recorded:

> reticulate:::miniconda_meta_path()
[1] "~/.local/share/r-reticulate/miniconda.json"

and when we check the content of that file

jovyan@jupyter-damianavila-402i2c-2eorg:~$ cat .local/share/r-reticulate/miniconda.json
{
  "DisableInstallationPrompt": true
}

Removing that file allowed me to see the miniconda prompt and I would bet this is how the user ended up in a setup where TF+Keras is not available.

yuvipanda commented 2 years ago

Yay!

2i2c-org / utoronto-image

Set RETICULATE_PYTHON env var #17