Publishing to ShinyApps.io with Python3

[ALaughingHorse]

I am trying to publish a shinyApp with Python3 to shinyApps.io. I added the following code in the app.R file as instructed by the Rstudio community. virtualenv_create('pyDev',python = 'python3') virtualenv_install("pyDev", packages = c('gensim', 'pandas','numpy','nltk','paramiko')) reticulate::use_virtualenv("pyDev", required = TRUE) However, when I publish the app, I got the following error:

Required version of Python '~/.virtualenvs/pyDev/bin/python' cannot be used because another version of Python ('/usr/bin/python') is already initialized for this process.

I can still open the published application, but it will "disconnect from server" whenever it tries to run python codes. I checked the log and the app is actually still running on Python2.7.

[BernoGreyling]

Having exactly the same issue. Also noticing the following messages in the log.

warning: using reticulate but python was not specified; will use python at /usr/bin/python Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

[ALaughingHorse]

@BernoGreyling Hi. For the message in your log, I had a similar one at first (using reticulate but python was not specified). But I was able to resolve it by using an older version of rsconnect. Run the code below to install the older package, restart the session and see if it helps you: devtools::install_github("rstudio/rsconnect", ref='737cd48')

[BernoGreyling]

Hey @ALaughingHorse. Already have that version installed. I see now I didn't mention that the issue above is on the shinyapps.io logs after deploying successfully.

[jmcphers]

Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

Can you try setting that variable in .rsconnect_profile? (see ?rsconnect::rsconnectOptions for more on where this file lives)

[ALaughingHorse]

@jmcphers In my .Rprofile, I have the following: Sys.setenv(RETICULATE_PYTHON='~/.virtualenvs/pyDev/bin/python')

In my app.R, I have:

virtualenv_create(envname = "pyDev",python = "python3") virtualenv_install(envname = "pyDev", packages = c('gensim', 'pandas','numpy','nltk'))

Am I specifying everything right??

I tried again, making sure that everything is included. This time, it seems that the app is published successfully. However, it disconnects immediately as it attempts to run the python file.

The log looks like below:

It seems that it's still using python 2.7 while trying to do the nltk download. I tried to set the Reticulate python to '~/.virtualenvs/pyDev/bin/python3', it will say the python executable cannot be found and point to the default python 2.7..

[jmcphers]

How are you deploying your app? If you are using the Publish button in RStudio, then setting the variable in your .Rprofile won't help since the publish happens in a background task. You need to set it in .rsconnect_profile for it to apply there. (This is a little confusing, and something we're hoping to improve!)

[BernoGreyling]

Hi @jmcphers. Is there a specific method to do this? I tried creating a .rsconnect_profile file in the app's root directory and setting the environmental variable there without success. The variable is set with Sys.setenv(RETICULATE_PYTHON='/usr/bin/python3') .

I'm on windows btw.

[rgavalda]

Hello. I am running into this same problem: shinyapps runs python 2.7 no matter how hard I try to direct it to pyhon3. I seem to have tried everything suggested in this issue (and elsewhere I could find). Can anybody report a success? (I need to use pandas and matplotlib, btw).

[peterhovmand]

I'm running into similar problems as well. Like others, I have tried many of the other solutions, but maybe I'm getting something basic wrong. In some posts elsewhere, it seems folks were able to solve the problem, but the solutions are not complete. It would be helpful if there is a complete example to draw from and test to understand what is going on when it's not working.

I can run Python code using reticulate's source_python() function to define and set a variable in Python that I can access and print in the ui, and this works both as a local execution and when I deploy to shinyapps.io. However, trying to create and use virtual environments only seems to work on my local machine, and I can't seem to successfully install Python packages using py_install() when deploying to shinyapps.io.

I'm using RStudio with the latest versions of shiny, reconnect, and reticulate.

[lassekv]

I just want to one-up this issue. I am also trying to use shinyapps.io with a Shiny application relying on Python modules used through reticulate. I have managed to get the application to run and install packages, but only with the Python 2.7 installation on the server. I have packages that require Python and so far I have not been able to get it to use the Python 3 installation. Using the latest rsconnect package it correctly fetches the RETICULATE_PYTHON variable from the .Renviron file within the project folder. This is however the reference on my local computer which does not make any sense on the shinyapps.io server. I had expected that rsconnect was using this variable to figure out which packages to install from the local environment within the shinyapps deployment, but that does not seem to happen... Any suggestions or examples for how to use Python 3 with shinyapps.io together with additional packages (e.g. pandas, elasticsearch and other standard pip-installable packages) would be much appreciated!

[dilan-jayasekara]

Did anyone had any luck solving this issue?

I'm facing the same problem.

[jspiewak]

See rstudio/reticulate#399 and give the development version a try.

[phoward38]

I'm using both of the same versions for reticulate & rsconnect as referenced in https://github.com/rstudio/reticulate/issues/399 but am still experiencing the same issue:

ERROR: The requested version of Python
('~/.virtualenvs/python_environment/bin/python') cannot be used, as
another version of Python ('/usr/bin/python') has already been
initialized. Please restart the R session if you need to attach
reticulate to a different version of Python.

[ranikay]

Since a lot of folks have commented on this issue, I wanted to point anyone still having difficulties to this end-to-end tutorial for deploying apps to shinyapps.io that use reticulate and Python 3. It shows how to configure virtual environments for use locally and on shinyapps.io, how to deploy a Shiny + reticulate app to shinyapps.io, how to confirm that your app deployed on shinyapps.io is using the desired version of Python, etc. I hope it's helpful for getting started!

[galen211]

Have spent the last couple hours trying to figure out the following error message on deploying with shinyapps.io: Did you forget to set the RETICULATE_PYTHON environment variable in your .Rprofile before publishing?

Per this thread, I understand that I need to set the python virtual environment in .rsconnect_profile, which I have done with the following:

RETICULATE_PYTHON="/Users/galen/Library/Caches/pypoetry/virtualenvs/financial-service-nwmTnczs-py3.7/"

Do I need to create a new virtual environment on the shinyapps.io server or does the deploy process capture the library dependencies in my local virtual environment? From the error message I can't tell whether my python dependencies are not loading because of something with the environment variables or something with the python libraries themselves. Specifically, the app starts locally but fails to start on shinyapps.io with the stacktrace below:

2020-04-28T17:22:21.369724+00:00 shinyapps[2201828]: Error in value[[3L]](cond) : ImportError: No module named 'grpc'

[kevinushey]

FWIW, I believe RETICULATE_PYTHON should point at the Python executable within that virtual environment, as opposed to the base directory of the virtual environment.

[rgavalda]

To add closure from my part, we moved to another environment without shinyapps so I never got to try ranikay's tutorial.

[galen211]

FWIW, I believe RETICULATE_PYTHON should point at the Python executable within that virtual environment, as opposed to the base directory of the virtual environment.

Thanks - I've tried pointing to /bin/python and same problem.

[pkansal83]

Since a lot of folks have commented on this issue, I wanted to point anyone still having difficulties to this end-to-end tutorial for deploying apps to shinyapps.io that use reticulate and Python 3. It shows how to configure virtual environments for use locally and on shinyapps.io, how to deploy a Shiny + reticulate app to shinyapps.io, how to confirm that your app deployed on shinyapps.io is using the desired version of Python, etc. I hope it's helpful for getting started!

Thankyou thankyou thankyou sooo much. Your code just is a magic and it worked for me. Thanks a lot :)

[nischalshrestha]

I have also encountered this problem in the learnr context, which might further illuminate why @ranikay's solution works. Something not explicitly mentioned in the end-to-end example is that using .Rprofile preemptively resolves the issue because RETICULATE_PYTHON is the first item reticulate checks for when loading a Python session. This is slightly different in the learnr context because I had to include library(reticulate) for the setup chunk as this is used by packrat (I believe) to install the package in the first place. But, this leads to the exact error OP is pointing out.

I tried solutions only using reticulate functions but failed because library(reticulate) started off with default the Python before I could do further configurations and use the correct Python virtualenv. If anyone has a better solution without using .Rprofile, let us know! But, it seems to do the job for now.

For those who are trying to get this to work in the learnr context, here's a sample file that might help (include the .Rprofile for deployment with appropriate values):

Example file

```` --- title: "Python reticulate" author: "Nischal Shrestha" output: learnr::tutorial runtime: shiny_prerendered --- ```{r setup, include=FALSE} library(reticulate) # required to install and load it on the server # If running locally, comment out # Precondition: .Rprofile has already executed to set environment variables when R session starts virtualenv_dir = Sys.getenv('VIRTUALENV_NAME') python_path = Sys.getenv('PYTHON_PATH') dependencies = c('numpy', 'pandas') reticulate::virtualenv_create(envname = virtualenv_dir, python = python_path) reticulate::virtualenv_install(virtualenv_dir, packages = dependencies) reticulate::use_virtualenv(virtualenv_dir, required = TRUE) library(learnr) ``` Test by importing the `mpg` dataset from R. We can do so with `r.mpg`: ```{python} df = r.mpg df ``` You should see a pandas.DataFrame being printed. ````

Please correct me if I am wrong on any of the details.

[hadley]

I think this is primarily a reticulate issue so I've asked @t-kalinowski to take a look.