plotly / Kaleido

Fast static image export for web-based visualization libraries with zero dependencies
MIT License
378 stars 39 forks source link

The kaleido executable is required by the kaleido Python library, but it was not included #153

Closed Antone4628 closed 4 months ago

Antone4628 commented 2 years ago

I installed Kaleido today using 'conda install -c conda-forge python-kaleido'. The environment I installed it to is running python 3.8.5.

When I attempted to export a Plotly plot I received the following error:

−2 −1.5 −1 −0.5 0 0.5 1 1.5 2 Dupin's Cyclide with Color Corresponding to Mean Curvature

ValueError Traceback (most recent call last) Cell In [19], line 24 21 if not os.path.exists("images"): 22 os.mkdir("images") ---> 24 sig.write_image("images/sig1.svg")

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/plotly/basedatatypes.py:3829, in BaseFigure.write_image(self, *args, *kwargs) 3769 """ 3770 Convert a figure to a static image and write it to a file or writeable 3771 object (...) 3825 None 3826 """ 3827 import plotly.io as pio -> 3829 return pio.write_image(self, args, **kwargs)

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/plotly/io/_kaleido.py:267, in write_image(fig, file, format, scale, width, height, validate, engine) 251 raise ValueError( 252 """ 253 Cannot infer image type from output path '{file}'. (...) 261 ) 262 ) 264 # Request image 265 # ------------- 266 # Do this first so we don't create a file if image conversion fails --> 267 img_data = to_image( 268 fig, 269 format=format, 270 scale=scale, 271 width=width, 272 height=height, 273 validate=validate, 274 engine=engine, 275 ) 277 # Open file 278 # --------- 279 if path is None: 280 # We previously failed to make sense of file as a pathlib object. 281 # Attempt to write to file as an open file descriptor.

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/plotly/io/_kaleido.py:144, in to_image(fig, format, width, height, scale, validate, engine) 141 # Validate figure 142 # --------------- 143 fig_dict = validate_coerce_fig_to_dict(fig, validate) --> 144 img_bytes = scope.transform( 145 fig_dict, format=format, width=width, height=height, scale=scale 146 ) 148 return img_bytes

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/scopes/plotly.py:153, in PlotlyScope.transform(self, figure, format, width, height, scale) 142 raise ValueError( 143 "Invalid format '{original_format}'.\n" 144 " Supported formats: {supported_formats_str}" (...) 148 ) 149 ) 151 # Transform in using _perform_transform rather than superclass so we can access the full 152 # response dict, including error codes. --> 153 response = self._perform_transform( 154 figure, format=format, width=width, height=height, scale=scale 155 ) 157 # Check for export error, later can customize error messages for plotly Python users 158 code = response.get("code", 0)

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/scopes/base.py:293, in BaseScope._perform_transform(self, data, **kwargs) 284 """ 285 Transform input data using the current scope, returning dict response with error code 286 whether successful or not. (...) 290 :return: Dict of response from Kaleido executable, whether successful or not 291 """ 292 # Ensure that kaleido subprocess is running --> 293 self._ensure_kaleido() 295 # Perform export 296 export_spec = self._json_dumps(dict(kwargs, data=data)).encode('utf-8')

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/scopes/base.py:176, in BaseScope._ensure_kaleido(self) 170 self._std_error = io.BytesIO() 172 # Launch kaleido subprocess 173 # Note: shell=True seems to be needed on Windows to handle executable path with 174 # spaces. The subprocess.Popen docs makes it sound like this shouldn't be 175 # necessary. --> 176 proc_args = self._build_proc_args() 177 self._proc = subprocess.Popen( 178 proc_args, 179 stdin=subprocess.PIPE, (...) 182 shell=sys.platform == "win32" 183 ) 185 # Set up thread to asynchronously collect standard error stream

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/scopes/base.py:123, in BaseScope._build_proc_args(self) 116 def _build_proc_args(self): 117 """ 118 Build list of kaleido command-line arguments based on current values of 119 the properties specified by self._scope_flags and self.chromium_args 120 121 :return: list of flags 122 """ --> 123 proc_args = [self.executable_path(), self.scope_name] 124 for k in self._scope_flags: 125 v = getattr(self, k)

File ~/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/scopes/base.py:99, in BaseScope.executable_path(cls) 97 path = os.environ.get("PATH", os.defpath) 98 formatted_path = path.replace(os.pathsep, "\n ") ---> 99 raise ValueError( 100 """ 101 The kaleido executable is required by the kaleido Python library, but it was not included 102 in the Python package and it could not be found on the system PATH. 103 104 Searched for included kaleido executable at: 105 {vendored_executable_path} 106 107 Searched for executable 'kaleido' on the following system PATH: 108 {formatted_path}\n""".format( 109 vendored_executable_path=vendored_executable_path, 110 formatted_path=formatted_path, 111 ) 112 ) 114 return executable_path

ValueError: The kaleido executable is required by the kaleido Python library, but it was not included in the Python package and it could not be found on the system PATH.

Searched for included kaleido executable at: /Users/antonechacartegui/opt/anaconda3/envs/pltenv/lib/python3.8/site-packages/kaleido/executable/kaleido

Searched for executable 'kaleido' on the following system PATH: /Users/antonechacartegui/opt/anaconda3/bin /Users/antonechacartegui/opt/anaconda3/condabin /usr/local/bin /usr/bin /bin /usr/sbin /sbin /Library/TeX/texbin /opt/X11/bin

Bewinxed commented 1 year ago

Having this issue as well, suddenly i'm getting this error for scripts that worked fine before

Antone4628 commented 1 year ago

I ended up installing Orca and using that engine to export my plots. It worked great for me. I'm not sure if this would help you for what you're doing but you might give it a try.

Bewinxed commented 1 year ago

For me, I uninstalled plotly and kaleido

reinstalled latest versions

then installed "urbansim" as some vague comment on StackOverflow suggested.

Now everything works :D

gregorywaynepower commented 1 year ago

For me, I uninstalled plotly and kaleido

reinstalled latest versions

then installed "urbansim" as some vague comment on StackOverflow suggested.

Now everything works :D

I did this, then attempting to install via pip as a last-ditch effort. But it ended up returning "kaleido already satisfied". And now I can generate images.

sigSquared2 commented 1 year ago

I'm using mac 10.14.6 with python 3.10 in a conda environment whose kernel is mapped to a running jupyter notebook from a separate base environment. After installing kaleido via conda install -c conda-forge python-kaleido into my conda environment and attempting to render a static image via plotly, I encountered the same error message referenced by the OP. What I found, however, is that an a executible was installed with the kaleido package under /Users/user_name/anaconda3/envs/env_name/bin. There are two other executibles in this same /bin directory, one is under /KaleidoApp, and the second under /KaleidoApp/bin. I'm unaware of the differences between all three, besides they each have different file sizes.

I generated a copy of the kaleido executible in the /env_name/bin directory and placed it in /Users/user_name/anaconda3/envs/env_name/lib/python3.10/site-packages/kaleido/executable, restarted my env_name kernel and everything was resolved. Note that I had to create the executable directory because it didn't exist. Perhaps this is a Kaleido issue, or maybe it has something to do with conda's package manager?

niketdoshi commented 1 year ago

I'm using mac 10.14.6 with python 3.10 in a conda environment whose kernel is mapped to a running jupyter notebook from a separate base environment. After installing kaleido via conda install -c conda-forge python-kaleido into my conda environment and attempting to render a static image via plotly, I encountered the same error message referenced by the OP. What I found, however, is that an a executible was installed with the kaleido package under /Users/user_name/anaconda3/envs/env_name/bin. There are two other executibles in this same /bin directory, one is under /KaleidoApp, and the second under /KaleidoApp/bin. I'm unaware of the differences between all three, besides they each have different file sizes.

I generated a copy of the kaleido executible in the /env_name/bin directory and placed it in /Users/user_name/anaconda3/envs/env_name/lib/python3.10/site-packages/kaleido/executable, restarted my env_name kernel and everything was resolved. Note that I had to create the executable directory because it didn't exist. Perhaps this is a Kaleido issue, or maybe it has something to do with conda's package manager?

It worked for me as well

creative-resort commented 1 year ago

The trouble is with the custom conda environment not being activated upon kernel launch, by standard. I've created an issue toward addressing this: [ENH] - include conda env activation for kernel with custom environment

njrhodes commented 9 months ago

I had to do two things to solve this problem fully:

First, when I tried running a minimally reproducible example using save_image() which calls kaleido, I prefaced with:

reticulate::py_run_string("import sys")

but this threw the error

ModuleNotFoundError: No module named 'rpytools'

I had to copy the rpytools folder from the reticulate installation (located at: C:\Users\nrhode\Documents\R\win-library\4.1\reticulate\python) to my virtual environment folder listed as number [8] below:

library(reticulate)
sys <- import("sys", convert = TRUE)
sys$path 
[1] ""                                                                                      
[2] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate"                    
[3] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\python39.zip"      
[4] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\DLLs"              
[5] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\lib"               
[6] "C:\\Program Files\\RStudio\\resources\\app\\bin"                                       
[7] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate"                    
[8] "C:\\Users\\nrhode\\AppData\\Local\\r-miniconda\\envs\\r-reticulate\\lib\\site-packages"

A clue came from: https://github.com/rstudio/keras/issues/615#issuecomment-856075119

This solved the first error.

>library(reticulate)
>sys <- import("sys", convert = TRUE)
>sys$path 
>import("rpytools")
Module(rpytools)

Next, I attempted a minimal example which threw an error when trying to find the kaleido executable:

p <- plotly::plot_ly(x = 1:10)
reticulate::py_run_string("import sys")
plotly::save_image(p, "./pic.png")

Error:

reticulate::py_last_error()

-- Python Exception Message ----------------------------------------------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\scopes\plotly.py", line 103, in transform
    response = self._perform_transform(
  File "C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\scopes\base.py", line 280, in _perform_transform
    self._ensure_kaleido()
  File "C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\scopes\base.py", line 166, in _ensure_kaleido
    proc_args = self._build_proc_args()
  File "C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\scopes\base.py", line 113, in _build_proc_args
    proc_args = [self.executable_path(), self.scope_name]
  File "C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\scopes\base.py", line 89, in executable_path
    raise ValueError(
ValueError: 
The kaleido executable is required by the kaleido Python library, but it was not included
in the Python package and it could not be found on the system PATH.

Searched for included kaleido executable at:
    C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate\lib\site-packages\kaleido\executable\kaleido 

Searched for executable 'kaleido' on the following system PATH:
...

I finally figured out where the Kaleido executables were located on my system (path below) AND that I had to add this path location explicitly to my Windows 10 Environmental variable path:

C:\Users\nrhode\AppData\Local\r-miniconda\pkgs\kaleido-core-0.1.0-h8ffe710_0\Library\bin\KaleidoApp

This solved the problem. @mnneely @siel

sessionInfo() R version 4.1.2 (2021-11-01) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale: [1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C
[5] LC_TIME=English_United States.1252

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] plotly_4.10.4 lubridate_1.9.3 forcats_1.0.0 stringr_1.5.1 dplyr_1.1.4
[6] purrr_1.0.2 readr_2.1.5 tidyr_1.3.1 tibble_3.2.1 ggplot2_3.5.0
[11] tidyverse_2.0.0 Pmetrics_2.1.1

loaded via a namespace (and not attached): [1] nlme_3.1-153 fs_1.6.3 usethis_2.1.6
[4] devtools_2.4.5 webshot_0.5.5 progress_1.2.3
[7] httr_1.4.7 rprojroot_2.0.3 numDeriv_2016.8-1.1
[10] backports_1.4.1 tools_4.1.2 profvis_0.3.8
[13] utf8_1.2.4 R6_2.5.1 DT_0.32
[16] metafor_4.2-0 lazyeval_0.2.2 colorspace_2.1-0
[19] ggdist_3.2.1 withr_3.0.0 urlchecker_1.0.1
[22] tidyselect_1.2.0 gridExtra_2.3 prettyunits_1.2.0
[25] processx_3.8.3 curl_5.2.0 compiler_4.1.2
[28] cli_3.6.2 scales_1.3.0 checkmate_2.3.1
[31] mvtnorm_1.2-4 callr_3.7.5 rappdirs_0.3.3
[34] trelliscopejs_0.2.6 digest_0.6.34 rmarkdown_2.25
[37] base64enc_0.1-3 pkgconfig_2.0.3 htmltools_0.5.7
[40] sessioninfo_1.2.2 fastmap_1.1.1 htmlwidgets_1.6.4
[43] rlang_1.1.3 rstudioapi_0.14 shiny_1.8.0
[46] farver_2.1.1 generics_0.1.3 jsonlite_1.8.8
[49] crosstalk_1.2.1 mclust_6.1 distributional_0.3.2
[52] magrittr_2.0.3 metadat_1.2-0 patchwork_1.2.0
[55] Matrix_1.5-4 Rcpp_1.0.12 munsell_0.5.0
[58] fansi_1.0.6 reticulate_1.35.0 lifecycle_1.0.4
[61] yaml_2.3.8 stringi_1.8.3 mathjaxr_1.6-0
[64] pkgbuild_1.4.0 grid_4.1.2 promises_1.2.1
[67] crayon_1.5.2 miniUI_0.1.1.1 lattice_0.20-45
[70] hms_1.1.3 knitr_1.45 ps_1.7.6
[73] pillar_1.9.0 codetools_0.2-18 pkgload_1.3.2
[76] glue_1.7.0 evaluate_0.23 DistributionUtils_0.6-1 [79] data.table_1.15.0 remotes_2.4.2 png_0.1-8
[82] vctrs_0.6.5 tzdb_0.4.0 httpuv_1.6.14
[85] foreach_1.5.2 gtable_0.3.4 cachem_1.0.8
[88] xfun_0.42 mime_0.12 xtable_1.8-4
[91] later_1.3.2 autocogs_0.1.4 viridisLite_0.4.2
[94] iterators_1.0.14 memoise_2.0.1 timechange_0.3.0
[97] ellipsis_0.3.2 here_1.0.1

reticulate::py_list_packages() package version requirement channel 1 blosc 1.21.5 blosc=1.21.5 conda-forge 2 bzip2 1.0.8 bzip2=1.0.8 conda-forge 3 c-blosc2 2.13.2 c-blosc2=2.13.2 conda-forge 4 ca-certificates 2024.2.2 ca-certificates=2024.2.2 conda-forge 5 hdf5 1.14.3 hdf5=1.14.3 conda-forge 6 intel-openmp 2024.0.0 intel-openmp=2024.0.0 conda-forge 7 kaleido-core 0.1.0 kaleido-core=0.1.0 conda-forge 8 krb5 1.21.2 krb5=1.21.2 conda-forge 9 libaec 1.1.2 libaec=1.1.2 conda-forge 10 libblas 3.9.0 libblas=3.9.0 conda-forge 11 libcblas 3.9.0 libcblas=3.9.0 conda-forge 12 libcurl 8.5.0 libcurl=8.5.0 conda-forge 13 libffi 3.4.2 libffi=3.4.2 conda-forge 14 libhwloc 2.9.3 libhwloc=2.9.3 conda-forge 15 libiconv 1.17 libiconv=1.17 conda-forge 16 liblapack 3.9.0 liblapack=3.9.0 conda-forge 17 libsqlite 3.45.1 libsqlite=3.45.1 conda-forge 18 libssh2 1.11.0 libssh2=1.11.0 conda-forge 19 libxml2 2.12.5 libxml2=2.12.5 conda-forge 20 libzlib 1.2.13 libzlib=1.2.13 conda-forge 21 lz4-c 1.9.4 lz4-c=1.9.4 conda-forge 22 mathjax 2.7.7 mathjax=2.7.7 conda-forge 23 mkl 2024.0.0 mkl=2024.0.0 conda-forge 24 numexpr 2.7.3 numexpr=2.7.3 conda-forge 25 numpy 1.26.4 numpy=1.26.4 conda-forge 26 openssl 3.2.1 openssl=3.2.1 conda-forge 27 orca 1.8 orca=1.8 conda-forge 28 packaging 23.2 packaging=23.2 conda-forge 29 pandas 2.2.1 pandas=2.2.1 conda-forge 30 patsy 0.5.6 patsy=0.5.6 conda-forge 31 pip 24.0 pip=24.0 conda-forge 32 plotly 5.19.0 plotly=5.19.0 plotly 33 prettytable 3.10.0 prettytable=3.10.0 conda-forge 34 pthreads-win32 2.9.1 pthreads-win32=2.9.1 conda-forge 35 py-cpuinfo 9.0.0 py-cpuinfo=9.0.0 conda-forge 36 pytables 3.9.2 pytables=3.9.2 conda-forge 37 python 3.9.18 python=3.9.18 conda-forge 38 python-dateutil 2.8.2 python-dateutil=2.8.2 conda-forge 39 python-kaleido 0.1.0 python-kaleido=0.1.0 conda-forge 40 python-tzdata 2024.1 python-tzdata=2024.1 conda-forge 41 python_abi 3.9 python_abi=3.9 conda-forge 42 pytz 2024.1 pytz=2024.1 conda-forge 43 pyyaml 6.0.1 pyyaml=6.0.1 conda-forge 44 scipy 1.12.0 scipy=1.12.0 conda-forge 45 setuptools 69.1.1 setuptools=69.1.1 conda-forge 46 six 1.16.0 six=1.16.0 conda-forge 47 snappy 1.1.10 snappy=1.1.10 conda-forge 48 statsmodels 0.14.1 statsmodels=0.14.1 conda-forge 49 tbb 2021.11.0 tbb=2021.11.0 conda-forge 50 tenacity 8.2.2 tenacity=8.2.2 pkgs/main 51 tk 8.6.13 tk=8.6.13 conda-forge 52 toolz 0.12.1 toolz=0.12.1 conda-forge 53 tzdata 2024a tzdata=2024a conda-forge 54 ucrt 10.0.22621.0 ucrt=10.0.22621.0 conda-forge 55 urbansim 3.2 urbansim=3.2 conda-forge 56 vc 14.3 vc=14.3 conda-forge 57 vc14_runtime 14.38.33130 vc14_runtime=14.38.33130 conda-forge 58 vs2015_runtime 14.38.33130 vs2015_runtime=14.38.33130 conda-forge 59 wcwidth 0.2.13 wcwidth=0.2.13 conda-forge 60 wheel 0.42.0 wheel=0.42.0 conda-forge 61 xz 5.2.6 xz=5.2.6 conda-forge 62 yaml 0.2.5 yaml=0.2.5 conda-forge 63 zlib-ng 2.0.7 zlib-ng=2.0.7 conda-forge 64 zstd 1.5.5 zstd=1.5.5 conda-forge

reticulate::conda_list() name 1 base 2 r-reticulate python 1 C:\Users\nrhode\AppData\Local\r-miniconda/python.exe 2 C:\Users\nrhode\AppData\Local\r-miniconda\envs\r-reticulate/python.exe

romaromaromann commented 6 months ago

I'm using mac 10.14.6 with python 3.10 in a conda environment whose kernel is mapped to a running jupyter notebook from a separate base environment. After installing kaleido via conda install -c conda-forge python-kaleido into my conda environment and attempting to render a static image via plotly, I encountered the same error message referenced by the OP. What I found, however, is that an a executible was installed with the kaleido package under /Users/user_name/anaconda3/envs/env_name/bin. There are two other executibles in this same /bin directory, one is under /KaleidoApp, and the second under /KaleidoApp/bin. I'm unaware of the differences between all three, besides they each have different file sizes.

I generated a copy of the kaleido executible in the /env_name/bin directory and placed it in /Users/user_name/anaconda3/envs/env_name/lib/python3.10/site-packages/kaleido/executable, restarted my env_name kernel and everything was resolved. Note that I had to create the executable directory because it didn't exist. Perhaps this is a Kaleido issue, or maybe it has something to do with conda's package manager?

Worked for me, thanks a lot

gvwilson commented 4 months ago

Hi - we are tidying up stale issues and PRs in Plotly's public repositories so that we can focus on things that are most important to our community. If this issue is still a concern, please add a comment letting us know what recent version of our software you've checked it with so that I can reopen it and add it to our backlog. (Please note that we will give priority to reports that include a short reproducible example.) If you'd like to submit a PR, we'd be happy to prioritize a review, and if it's a request for tech support, please post in our community forum. Thank you - @gvwilson

TheMadMoose commented 2 months ago

I'm not sure if it's okay that i'm posting this (my apologies otherwise). I encounted a similar issue and figured it might be worth it to let someone know in case they're in the same boat. I started from the lasted version 0.2.1/post1 and went downwards until it worked with 0.1.0.post1 on windows 11.

https://github.com/plotly/Kaleido/issues/61#issuecomment-2338125494