search

rstudio / reticulate

R Interface to Python
https://rstudio.github.io/reticulate
Apache License 2.0
1.67k stars 327 forks source link

boto3: Error: Unable to locate credentials #744

Open DyfanJones opened 4 years ago

DyfanJones commented 4 years ago

Hi all,

I am currently developing a package that utilises reticulate to interface with the python package boto3 to make a connection to Athena.

Recently a user raised an issue where credentials weren't getting retrieved by reticulate when making a boto3 connection: https://github.com/DyfanJones/RAthena/issues/98 .

The user highlight that the python code runs successful and fails when using the reticulate wrapper.

Python

import boto3

session = boto3.Session()
credentials = session.get_credentials().get_frozen_credentials()
credentials

reticulate

library(reticulate)
boto3 <- import("boto3")
session <- boto3$Session()
credentials <- session$get_credentials()$get_frozen_credentials()
credentials

Error: attempt to apply non-function

User R Session Info:

Session Info:

Session info --------------------------------------------------------------- setting value version R version 3.5.1 (2018-07-02) os Windows 10 x64 system x86_64, mingw32 ui RTerm language (EN) collate English_United States.1252 ctype English_United States.1252 tz Asia/Jerusalem date 2020-03-26

Packages ------------------------------------------------------------------- ! package version date lib source arrow 0.16.0.2 2020-02-14 [1] CRAN (R 3.5.3) assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.5.3) AWR.Athena 2.0.7-0 2019-07-08 [1] CRAN (R 3.5.3) aws.s3 0.3.20 2020-03-11 [1] CRAN (R 3.5.3) aws.signature 0.5.2 2019-08-08 [1] CRAN (R 3.5.3) backports 1.1.5 2019-10-02 [1] CRAN (R 3.5.3) base64enc 0.1-3 2015-07-28 [1] CRAN (R 3.5.2) bit 1.1-15.2 2020-02-10 [1] CRAN (R 3.5.3) bit64 0.9-7 2017-05-08 [1] CRAN (R 3.5.3) callr 3.4.1 2020-01-24 [1] CRAN (R 3.5.3) cli 2.0.0 2019-12-09 [1] CRAN (R 3.5.3) crayon 1.3.4 2017-09-16 [1] CRAN (R 3.5.3) curl 4.3 2019-12-02 [1] CRAN (R 3.5.3) data.table 1.12.8 2019-12-09 [1] CRAN (R 3.5.3) DBI 1.1.0.9000 2020-03-04 [1] Github (r-dbi/DBI@500edd3) desc 1.2.0 2018-05-01 [1] CRAN (R 3.5.3) devtools 2.2.2 2020-02-17 [1] CRAN (R 3.5.3) digest 0.6.23 2019-11-23 [1] CRAN (R 3.5.3) ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.5.3) evaluate 0.14 2019-05-28 [1] CRAN (R 3.5.3) fansi 0.4.0 2018-10-05 [1] CRAN (R 3.5.3) fs 1.3.1 2019-05-06 [1] CRAN (R 3.5.3) glue 1.3.1 2019-03-12 [1] CRAN (R 3.5.3) htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.5.3) httr 1.4.1 2019-08-05 [1] CRAN (R 3.5.3) IRdisplay 0.7.0 2018-11-29 [1] CRAN (R 3.5.3) IRkernel 1.1 2019-12-06 [1] CRAN (R 3.5.3) jsonlite 1.6.1 2020-02-02 [1] CRAN (R 3.5.3) magrittr 1.5 2014-11-22 [1] CRAN (R 3.5.3) memoise 1.1.0 2017-04-21 [1] CRAN (R 3.5.3) pbdZMQ 0.3-3 2018-05-05 [1] CRAN (R 3.5.2) pillar 1.4.3 2019-12-20 [1] CRAN (R 3.5.3) pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.5.3) pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.5.3) prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.5.3) processx 3.4.2 2020-02-09 [1] CRAN (R 3.5.3) ps 1.3.1 2020-02-12 [1] CRAN (R 3.5.1) purrr 0.3.3 2019-10-18 [1] CRAN (R 3.5.3) R6 2.4.1 2019-11-12 [1] CRAN (R 3.5.3) Rcpp 1.0.3 2019-11-08 [1] CRAN (R 3.5.3) remotes 2.1.1 2020-02-15 [1] CRAN (R 3.5.3) repr 1.0.2 2019-12-16 [1] CRAN (R 3.5.3) D rJava 0.9-11 2019-03-29 [1] CRAN (R 3.5.3) RJDBC 0.2-8 2020-03-10 [1] CRAN (R 3.5.3) rlang 0.4.5 2020-03-01 [1] CRAN (R 3.5.3) rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.5.3) sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.5.3) stringi 1.4.6 2020-02-17 [1] CRAN (R 3.5.3) testthat 2.3.1 2019-12-01 [1] CRAN (R 3.5.3) tidyselect 1.0.0 2020-01-27 [1] CRAN (R 3.5.3) usethis 1.5.1 2019-07-04 [1] CRAN (R 3.5.3) uuid 0.1-2 2015-07-28 [1] CRAN (R 3.5.2) vctrs 0.2.4 2020-03-10 [1] CRAN (R 3.5.3) withr 2.1.2 2018-03-15 [1] CRAN (R 3.5.3) xml2 1.2.2 2019-08-09 [1] CRAN (R 3.5.3)

kevinushey commented 4 years ago 
> creds <- session$get_credentials()
> creds
NULL

Are you sure this is retrieving the expected credentials? I don't know how this is supposed to be configured.

DavidArenburg commented 4 years ago

Basically, it seems that reticulate doesn't know how to check environment variables (in case Python wasn't installed via Anaconda).

image

The only way I found around it (which doesn't make much sense to me - why not check the environment variables as default?), is to set up a specific environment variable for reticulate as I've mentioned here https://github.com/DyfanJones/RAthena/issues/98#issuecomment-604539141

In either way, it seems that this still keeps messing up different stuff, as the above mentioned issue.

UjwalKavalipati commented 3 years ago

Any update on this issue?

AaronNHart commented 2 years ago

I also seem to be experiencing this issue.

I just noticed that when running boto3 <- import("boto3") from reticulate, that the first time I make this call it asks if I want to install miniconda. Maybe this prompt is causing the exception somehow?

t-kalinowski commented 2 years ago

Looking over this thread the issue seems to be stemming from reticulate not bind to the appropriate Python installation. If reticulate is suggesting installation of miniconda, that means that it's searched through all the Python installations it knows about for the requested module, ('boto3' here), not found it, and is now looking for a 'default' python to bind to.

If the auto-discover mechanism that reticulate is using to pick a Python installation isn't working for you, you can be deliberate and explicitly tell reticulate which Python to use:

library(reticulate)
use_python("/path/to/bin/python")
py_config() # forces reticulate to bind to the python

Once reticulate has bound to a python installation, py_install() will default to that installation, e.g: py_install("boto3")