Azure / azureml-sdk-for-r

Azure Machine Learning SDK for R
https://azure.github.io/azureml-sdk-for-r/
Other
104 stars 40 forks source link

Cannot download output files from Run #406

Closed michalmar closed 3 years ago

michalmar commented 3 years ago

Describe the bug I cannot download files (model) generated by training experiment - it is in outputs folder. On Jupyter I am getting "Kernel died" error and no details.

To Reproduce Train model within an Experiment and upload some data (model) to output's folder. I am basically following tutorial from vignette: https://azure.github.io/azureml-sdk-for-r/articles/train-and-deploy-first-model.html

Expected behavior data will be downloaded

image

revodavid commented 3 years ago

I am having a related issue. This command, which worked as of August, now produces an error:

> download_files_from_run(run, prefix="outputs")
Error: C stack usage  1060675652244 is too close to the limit

download_file_from_run fails with a similar error

revodavid commented 3 years ago

Updating to the latest version of reticulate from github may resolve this for you: https://github.com/rstudio/reticulate

michalmar commented 3 years ago

@revodavid thanks for hint - it hasn't resolved my issue though: I still have the same error.

btw my env is:

R version 3.6.3 (2020-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04.7 LTS

Matrix products: default
BLAS/LAPACK: /opt/intel/compilers_and_libraries_2018.2.199/linux/mkl/lib/intel64_lin/libmkl_rt.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8       
 [4] LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C              
[10] LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.18   azuremlsdk_1.10.0

loaded via a namespace (and not attached):
[1] compiler_3.6.3  Matrix_1.2-18   tools_3.6.3     rappdirs_0.3.1  Rcpp_1.0.5      grid_3.6.3     
[7] jsonlite_1.7.1  lattice_0.20-41
revodavid commented 3 years ago

I'm not 100% sure, but it looks like you are using the CRAN version of reticulate. The version on GitHub is 1.18-9000.

michalmar commented 3 years ago

Any recommendations on how to install/use the right version then, please?

diondrapeck commented 3 years ago

To install the latest github version, use this: remotes::install_github('https://github.com/Azure/azureml-sdk-for-r')

adfi commented 3 years ago

I have the same issue, similar environment as @michalmar.

@diondrapeck are you suggesting we update the azureml package? the suggestion from @revodavid was to update reticulate (which I have done). I can update azureml locally but what about the environment running the training code?

adfi commented 3 years ago

FYI, I tried using a compute cluster instead of 'local' as the compute target and that did result in an outputs folder from which I could download model files.

diondrapeck commented 3 years ago

@adfi - my apologies! I misunderstood your question. The latest github version of reticulate would of course be the same command as above with "https://github.com/rstudio/reticulate" substituted for the azure ml sdk url.

I'm glad that you were able to get an outputs folder by switched to compute cluster.