how to use reticulate when python package not available on anaconda

gadepallivs commented 5 years ago

I am attempting to work with SMART on FHIR with python FHIR client. It works as expected after installing pip install fhirclient and running the example on python command line.

However, I am stuck with the first step. I get an error ModuleNotFoundError: No module named 'fhirclient'. Anaconda does not have this fhirclient. So, wondering if there is a way I can get this working.

my python test code in a separate file (fhir_test.py). These command run on python command lines without errors and desired output is shown. FYI there is a package called RonFHIR for similar task of pulling fhir data into R, however it seems that python fhirclient is more popular in use, so wanted to try and see if reticulate package helps me to pull data into R

from fhirclient import client
import fhirclient.models.patient as p

settings = {'app_id': 'cerner1',
            'api_base': 'https://r3.smarthealthit.org'}
smart = client.FHIRClient(settings=settings)
patient = p.Patient.read('fdb4e56a-145b-4962-8a54-e056757832aa',
                         smart.server)

The r code


library(reticulate)
use_python(Sys.which("python"))
#py_config()
#py_discover_config()
py_install("fhirclient")
#> Error: Error 1 occurred installing packages into conda environment r-reticulate
py_module_available("fhirclient")
#> [1] FALSE
source_python("fhir_test.py")
#> Error in py_run_file_impl(file, local, convert) :  
#>  ModuleNotFoundError: No module named 'fhirclient' 
#>  Detailed traceback:   File "<string>", line 1, in <module>

^{Created on 2019-08-13 by the reprex package (v0.3.0)}

Session info

``` r devtools::session_info() #> - Session info ---------------------------------------------------------- #> setting value #> version R version 3.5.3 (2019-03-11) #> os Windows 10 x64 #> system x86_64, mingw32 #> ui RTerm #> language (EN) #> collate English_United States.1252 #> ctype English_United States.1252 #> tz America/New_York #> date 2019-08-13 #> #> - Packages -------------------------------------------------------------- #> package * version date lib source #> assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.3) #> backports 1.1.4 2019-04-10 [1] CRAN (R 3.5.3) #> callr 3.3.1 2019-07-18 [1] CRAN (R 3.5.3) #> cli 1.1.0 2019-03-19 [1] CRAN (R 3.5.3) #> crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.3) #> desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.3) #> devtools 2.1.0 2019-07-06 [1] CRAN (R 3.5.3) #> digest 0.6.20 2019-07-04 [1] CRAN (R 3.5.3) #> evaluate 0.14 2019-05-28 [1] CRAN (R 3.5.3) #> fs 1.3.1 2019-05-06 [1] CRAN (R 3.5.3) #> glue 1.3.1 2019-03-12 [1] CRAN (R 3.5.3) #> highr 0.8 2019-03-20 [1] CRAN (R 3.5.3) #> htmltools 0.3.6 2017-04-28 [1] CRAN (R 3.5.3) #> jsonlite 1.6 2018-12-07 [1] CRAN (R 3.5.3) #> knitr 1.23 2019-05-18 [1] CRAN (R 3.5.3) #> lattice 0.20-38 2018-11-04 [1] CRAN (R 3.5.3) #> magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.3) #> Matrix 1.2-17 2019-03-22 [1] CRAN (R 3.5.3) #> memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.3) #> pkgbuild 1.0.3 2019-03-20 [1] CRAN (R 3.5.3) #> pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.3) #> prettyunits 1.0.2 2015-07-13 [1] CRAN (R 3.5.3) #> processx 3.4.1 2019-07-18 [1] CRAN (R 3.5.3) #> ps 1.3.0 2018-12-21 [1] CRAN (R 3.5.3) #> R6 2.4.0 2019-02-14 [1] CRAN (R 3.5.3) #> Rcpp 1.0.2 2019-07-25 [1] CRAN (R 3.5.3) #> remotes 2.1.0 2019-06-24 [1] CRAN (R 3.5.3) #> reticulate * 1.13 2019-07-24 [1] CRAN (R 3.5.3) #> rlang 0.4.0 2019-06-25 [1] CRAN (R 3.5.3) #> rmarkdown 1.14 2019-07-12 [1] CRAN (R 3.5.3) #> rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.3) #> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.3) #> stringi 1.4.3 2019-03-12 [1] CRAN (R 3.5.3) #> stringr 1.4.0 2019-02-10 [1] CRAN (R 3.5.3) #> testthat 2.2.1 2019-07-25 [1] CRAN (R 3.5.3) #> usethis 1.5.1 2019-07-04 [1] CRAN (R 3.5.3) #> withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.3) #> xfun 0.8 2019-06-25 [1] CRAN (R 3.5.3) #> yaml 2.2.0 2018-07-25 [1] CRAN (R 3.5.3) #> #> [1] C:/Users/gade04/Documents/R/R-3.5.3/library ```

kevinushey commented 5 years ago

You can use py_install("fhirclient", pip = TRUE) to force installation with pip as opposed to conda.

gadepallivs commented 5 years ago

It throws the following error. and the method is auto, options are virtulaenv or conda. I guess, it is picking conda by default here.

py_install("numpy", pip = TRUE)
Error: Conda installation not found (failed to locate conda binary)
Please install Anaconda for Windows (https://www.anaconda.com/download/#windows)
before proceeding.

However, if I sidestep by directly opening windows powershell and then install the package pip install fhirclient it works

py_module_available("fhirclient")
[1] TRUE

kevinushey commented 5 years ago

Do you have Anaconda installed? On Windows, py_install() is mainly tooled for working with Python environments (conda on Windows; virtualenvs on other platforms)

gadepallivs commented 5 years ago

Yes, I have Anaconda installed. If I try py_install with pip = TRUE & with anaconda installed on the computer. The pip = TRUE parameter does force install fhirclient without errors ( could not locate the directories the modules were installed) . However, It still throws the error ModuleNotFoundError: No module named fhirclient. And, when I check py_module_available("fhirclient") it outputs FALSE. May be the directories path needs to be set for python to access modules installed using py_install ?. Uninstalling anaconda and installing fhirclient on command line, I was able to load the fhirclinet module and run within reticulate package.

cderv commented 5 years ago

When selecting a working condaenv, this works as expected

library(reticulate)
use_condaenv("r-reticulate")
py_install("fhirclient", pip = TRUE)
py_module_available("fhirclient")
fh <- import("fhirclient")

absognety commented 4 years ago

When selecting a working condaenv, this works as expected

library(reticulate)
use_condaenv("r-reticulate")
py_install("fhirclient", pip = TRUE)
py_module_available("fhirclient")
fh <- import("fhirclient")

This worked

Prometheus77 commented 4 years ago

When selecting a working condaenv, this works as expected
library(reticulate)
use_condaenv("r-reticulate")
py_install("fhirclient", pip = TRUE)
py_module_available("fhirclient")
fh <- import("fhirclient")
This worked

This didn't work. Everything seems to install, but when I get to py_module_available('fhirclient'), I get FALSE.

absognety commented 4 years ago

When selecting a working condaenv, this works as expected
library(reticulate)
use_condaenv("r-reticulate")
py_install("fhirclient", pip = TRUE)
py_module_available("fhirclient")
fh <- import("fhirclient")
This worked
This didn't work. Everything seems to install, but when I get to py_module_available('fhirclient'), I get FALSE.

@Prometheus77, python you are using must have been from different environment from the environment in which r-reticulate is installed.
Do:

conda_list() to see all available conda envs
use_python("/home/vikas/.local/share/r-miniconda/envs/r-reticulate/bin/python") to change the python to correct one in which r-reticulate is setup
Sys.which("python") to see if it is changed properly

Then proceed to installing package you want with py_install("fhirclient", pip = TRUE) and see py_module_available("fhirclient"). everything is illustrated below:

> library(reticulate)
> use_condaenv("r-reticulate")
> py_install("fhirclient", pip = TRUE)

EnvironmentLocationNotFound: Not a conda environment: /home/vikas/anaconda3/envs/r-reticulate

Error: installation of 'python' into environment 'r-reticulate' failed [error code 1]
> Sys.which("python")
                            python 
"/home/vikas/anaconda3/bin/python" 
> py_module_available("fhirclient")
[1] FALSE
> conda_list()
          name
1  r-miniconda
2 r-reticulate
3    anaconda3
                                                             python
1                   /home/vikas/.local/share/r-miniconda/bin/python
2 /home/vikas/.local/share/r-miniconda/envs/r-reticulate/bin/python
3                                  /home/vikas/anaconda3/bin/python
> use_python
function (python, required = FALSE) 
{
    if (required && !file_test("-f", python) && !file_test("-d", 
        python)) 
        stop("Specified version of python '", python, "' does not exist.")
    if (required && is_python_initialized()) {
        if (!file_same(py_config()$python, python)) {
            fmt <- paste("ERROR: The requested version of Python ('%s') cannot be used, as", 
                "another version of Python ('%s') has already been initialized.", 
                "Please restart the R session if you need to attach reticulate", 
                "to a different version of Python.")
            msg <- sprintf(fmt, python, py_config()$python)
            writeLines(strwrap(msg), con = stderr())
            stop("failed to initialize requested version of Python")
        }
    }
    if (required) 
        .globals$required_python_version <- python
    .globals$use_python_versions <- unique(c(.globals$use_python_versions, 
        python))
}
<bytecode: 0x56000d7a7090>
<environment: namespace:reticulate>
> use_python("/home/vikas/.local/share/r-miniconda/envs/r-reticulate/bin/python")
> Sys.which("python")
                                                             python 
"/home/vikas/.local/share/r-miniconda/envs/r-reticulate/bin/python" 
> py_module_available("fhirclient")
[1] FALSE
> py_install("fhirclient", pip = TRUE)
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
  current version: 4.8.3
  latest version: 4.8.4

Please update conda by running

    $ conda update -n base -c defaults conda

## Package Plan ##

  environment location: /home/vikas/.local/share/r-miniconda/envs/r-reticulate

  added / updated specs:
    - python

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2020.6.24  |                0         125 KB
    ------------------------------------------------------------
                                           Total:         125 KB

The following packages will be UPDATED:

  ca-certificates                                2020.1.1-0 --> 2020.6.24-0

Downloading and Extracting Packages
ca-certificates-2020 | 125 KB    | ################################################################################################### | 100% 
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
Collecting fhirclient
  Downloading fhirclient-3.2.0-py2.py3-none-any.whl (550 kB)
     |████████████████████████████████| 550 kB 98 kB/s 
Collecting isodate
  Using cached isodate-0.6.0-py2.py3-none-any.whl (45 kB)
Collecting requests
  Using cached requests-2.24.0-py2.py3-none-any.whl (61 kB)
Requirement already satisfied, skipping upgrade: six in /home/vikas/.local/lib/python3.6/site-packages (from isodate->fhirclient) (1.14.0)
Collecting idna<3,>=2.5
  Using cached idna-2.10-py2.py3-none-any.whl (58 kB)
Collecting chardet<4,>=3.0.2
  Using cached chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /home/vikas/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests->fhirclient) (2020.6.20)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /home/vikas/.local/share/r-miniconda/envs/r-reticulate/lib/python3.6/site-packages (from requests->fhirclient) (1.25.9)
Installing collected packages: isodate, idna, chardet, requests, fhirclient
Successfully installed chardet-3.0.4 fhirclient-3.2.0 idna-2.10 isodate-0.6.0 requests-2.24.0
> pip_list()
Error in pip_list() : could not find function "pip_list"
> py_module_available("fhirclient")
[1] TRUE
> Sys.which("python")
                                                             python 
"/home/vikas/.local/share/r-miniconda/envs/r-reticulate/bin/python" 
> pd
Error: object 'pd' not found
> pd <- import("pandas")
> pd
Module(pandas)
> fclient <- import("fhirclient")
> fclient
Module(fhirclient)

rstudio / reticulate

how to use reticulate when python package not available on anaconda #578