RStudio crashes when browsing certain methods

[thohan88]

First of all, thank you so much for a fantastic package!

I am using Reticulate to call a Python Storage-module for Azure. This works perfectly. However, I have discovered that RStudio crashes for certain methods when they are displayed in the dropdown menu. In my case, it seem to happen only for methods containing the word "copy". Browsing copy-methods from the azure module crashes RStudio (example 1 and 2). Browsing copy-methods from the shutil-module gives an error message (example 3). I am unable to reproduce this for copy-methods from pythons copy-module which are displayed correctly.

I have tried both the CRAN-version and Github-version of reticulate. Please see attached examples.

Example 1 (Crashes when browsing CopyProperties of asb)

library(reticulate)
asb  <- import("azure.storage.blob")

# This is the method which crashes when browsing
asb$CopyProperties

Example 2 (Crashes when browsing method copy_blob)

library(reticulate)

asb <- import("azure.storage.blob")

# Setup credentials (Disguised for the bug report)
#account_name <- '###'
#account_key  <- '###'

# Create blob object
bb <- asb$BlockBlobService(account_name = account_name, account_key = account_key)

# This is the method which crashes when browsing
bb$copy_blob

Example 3 (Does not actually crash RStudio but gives an error)

library(reticulate)

shutil <- import("shutil")

# This is the method which crashes when browsing
shutil$copy

Returns the following error when browsing shutil$copy

Error in py_call_impl(callable, dots$args, dots$keywords) : 
  ValueError: Function has keyword-only parameters or annotations, use getfullargspec() API which can support them

Session info

R version 3.3.3 (2017-03-06)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1

locale:
[1] LC_COLLATE=Norwegian (Bokmål)_Norway.1252  LC_CTYPE=Norwegian (Bokmål)_Norway.1252   
[3] LC_MONETARY=Norwegian (Bokmål)_Norway.1252 LC_NUMERIC=C                              
[5] LC_TIME=Norwegian (Bokmål)_Norway.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_0.7.0.9002

loaded via a namespace (and not attached):
[1] tools_3.3.3  Rcpp_0.12.10

[jjallaire]

What version of Python are you using. I have so far been unable to reproduce on OS X running Anaconda Python 3.6. Will try on Windows shortly...

[jjallaire]

Darnit, also unable to reproduce running Python 2.7 on Windows :-\

@dfalty could you work w/ @thohan88 to get a reproducible scenario here?

[ericwatt]

I'm getting the same error in Windows 10 with a conda environment at Python 3.5.3. I haven't had a full crash, but it spits out the following message similar to example 3 above. Currently using reticulate 0.7 from CRAN. R 3.3.3 RStudio 1.0.141.

Error in py_call_impl(callable, dots$args, dots$keywords) : 
    ValueError: Function has keyword-only arguments or annotations, use getfullargspec() API which can support them

[jjallaire]

Okay, I have repro'd both issues under Anaconda Python 3.6 on Windows. I have a fix for the error message on shutil$copy (just checked in) but not yet for the crash.

[ericwatt]

I just installed the newest commit to see if this fixed it for me, but now I'm getting a new error. Not sure if this is related or not, let me know if you would like it moved to a new issue.

library(reticulate)
os <- import("os")

Error in py_initialize(config$python, config$libpython, config$pythonhome,  : 
  expecting a string

It's able to find python however:

> py_discover_config()
python:         C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1\python.exe
libpython:      C:/Users/watte/AppData/Local/CONTIN~1/ANACON~1/python36.dll
pythonhome:     C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1
version:        3.6.0 |Anaconda 4.3.1 (64-bit)| (default, Dec 23 2016, 11:57:41) [MSC v.1900 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1\lib\site-packages\numpy
numpy_version:  1.11.3

> use_condaenv("cadme", required = TRUE)
> py_discover_config()
python:         C:\Users\watte\AppData\Local\Continuum\Anaconda3\envs\cadme\python.exe
libpython:      C:/Users/watte/AppData/Local/Continuum/Anaconda3/envs/cadme/python35.dll
pythonhome:     C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1\envs\cadme
version:        3.5.3 |Continuum Analytics, Inc.| (default, Feb 22 2017, 21:28:42) [MSC v.1900 64 bit (AMD64)]
Architecture:   64bit
numpy:          C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1\envs\cadme\lib\site-packages\numpy
numpy_version:  1.12.1

python versions found: 
 C:\Users\watte\AppData\Local\Continuum\Anaconda3\envs\cadme\python.exe
 C:\Users\watte\AppData\Local\CONTIN~1\ANACON~1\python.exe

[jjallaire]

I've got the code required to reproduce the crash down to this:

library(reticulate)
asb <- import("azure.storage.blob")
py_get_attr(asb$CopyProperties, "__doc__")

Note that getting other attributes besides "doc" on the object (e.g. "module" or "class") work fine. There is something about the "doc" that is causing problems.

[thohan88]

Thanks for the quick response. It is indeed Anaconda Python 3.6.0 with azure-storage directly from git (v. 0.34.0). The latest commit solved the shdir-example. I could also run the code of @ericwatt without any issues.

py_get_attr(asb$CopyProperties, "__doc__") does indeed reproduce the crash. I wish I could be of more help in finding the cause.

[thohan88]

I think I have located the cause. On line 208 of models.py, the documentation of CopyProperties includes the character ’ in don’t. If I replace it with ' I get no errors.

So changing line 208:

                non-fatal errors impede copy progress but don’t cause failure.

to:

                non-fatal errors impede copy progress but don't cause failure.

does not crash RStudio and displays CopyProperties beautifully. I tried a non-exhaustive list of other common special characters \\^$|~!@#$%^&*(){}_+:"<>?/;'[]-= which all worked fine. It appears to be an exclusive issue of the ’-character

Other copy-functions of azure-storage return parsed CopyProperties with the same doc-issue which confused me into thinking it was related to the copy-prefix.

[jjallaire]

That's fantastic sleuthing, thank you so much!!!!

I have a fix here: https://github.com/rstudio/reticulate/commit/39ae098ecde4d60146c36cc3ad1017652e01239d

Let me know if this gives you any more trouble!

[thohan88]

Works perfectly now, thank you!