Image generation hangs on windows 10

[ammaar8]

Hey, I'm trying to convert figures into static images but it refuses to work on windows 10. I am using this script for testing -

import plotly.graph_objects as go
import numpy as np
import plotly.io as pio
np.random.seed(1)

N = 100
x = np.random.rand(N)
y = np.random.rand(N)
colors = np.random.rand(N)
sz = np.random.rand(N) * 30

fig = go.Figure()
fig.add_trace(go.Scatter(
    x=x,
    y=y,
    mode="markers",
    marker=go.scatter.Marker(
        size=sz,
        color=colors,
        opacity=0.6,
        colorscale="Viridis"
    )
))

fig.write_image('test.png')
fig.show()

It just hangs forever. I tried with both kaleido and orca, same thing. I tried to find out where it was hanging using pycharm and it seems to be this line - https://github.com/plotly/Kaleido/blob/6a46ecae926b4c004bf7232383cf7c74c70748fd/repos/kaleido/py/kaleido/scopes/base.py#L192 If you require anymore info let me know.

[ghost]

Having the same issue 😢.

[ammaar8]

Having the same issue 😢.

If it's any help, I am now using wsl2 with ubuntu 20 for this. If it doesn't work on localhost, you can host it on 0.0.0.0 and view it on the wsl's ip. Hoping this gets fixed soon since running it on wsl is a lot slower but have no other option.

[jonmmease]

If possible, it would be helpful to know if the instructions in this comment produce any additional logging info: https://github.com/plotly/Kaleido/issues/36#issuecomment-756676527

[ammaar8]

If possible, it would be helpful to know if the instructions in this comment produce any additional logging info: #36 (comment)

I tried doing it in cmd and jupyter notebook but it won't interrupt using ctrl+c/interrupt kernel. Any other ideas?

[jonmmease]

Ok, thanks for giving that a try. The next idea is to try to run the kaleido executable CMD, without python.

From python, run this to print out the (long) kaleido launch command

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

Then run this from the CMD prompt. If successful, it would print out

{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}

along with some benign warnings. But if it hangs here, it would be great to record what is logged.

[ammaar8]

[0922/185509.685:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale
[0922/185509.743:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}
[0922/185510.510:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

Sorry about the late reply, this is what I get.

[leeTimothy]

+1 for similar issue with similar output as above

[jonmmease]

Ok, thanks @ammaar8 and @leeTimothy. That's helpful.

It's possible that the key error is

[0922/185510.510:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

Although the {"code: 0,...} line indicates that kaleido did launch successfully.

After running that first command, could you try pasting the following content into the console and pressing Enter:

{"data":{"layout": {}}, "format":"svg"}

This is an actual image export request, and if successful would print out an SVG string. Again, it would be helpful to know if this prints out any warnings or errors, or if it hangs.

Thanks!

[leeTimothy]

this might be where the behaviour im experiencing might deviate from @ammaar8

although running the output of

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

provides a very similar trace like ammaar

[0924/131936.214:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale 
[0924/131936.234:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}
[0924/131936.291:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader

the command itself never runs to completion and just hangs there.

if the behaviour deviates - please let me know if you want me to raise this separately.

cheers

[jonmmease]

@leeTimothy, the first command is intended to stay running and wait for image export requests. Did you try pasting the JSON snippet I described above and hitting enter? It would be good to know if this hangs, or if any additional info is logged. Thanks!

[leeTimothy]

Ah apologies!

Please see below.

{"data":{"layout": {}}, "format":"svg"}
{"code": 1, "message": "Failed to serialize document: Uncaught", "result": null, "version": "0.2.1"}

[jonmmease]

Ok, thanks @leeTimothy.

Here are a couple of experiments to try. The long kaleido command has a bunch of chromium flags. e.g. one of them is --disable-gpu.

1. Try deleting the --disable-gpu flag, and performing the test process again.
2. Try adding the --disable-webgl flag.

If you, or anyone else, is interested in experimenting more with chromium options. Here's a giant list of the available chromium flags: https://peter.sh/experiments/chromium-command-line-switches/.

[leeTimothy]

Hey Jon,

Apols for the late response - believe I may have narrowed it down slightly more. Issue appears to be specific to Windows machines as haven't seen any other reports on the issuse for anything else.

I've got access to 3 different W10 machines, 2 Running OS Build 19043+ both of which Kaleido doesn't currently work. The W10 Machine running on OS Build 19041 is still able to properly export static images.

@ammaar8 @guilhermebarrett can you confirm what OS build of W10 you're currently running?

[ammaar8]

Hey @leeTimothy,

Sorry for the late reply, I'm on 19043.1237 like you are!

[jonmmease]

That version info helps a lot, thanks for investigating @leeTimothy and @ammaar8

[ammaar8]

@jonmmease I wasn't able to paste the string in the console.

[RyanChapman2x]

Just to add: I have three Windows 10 machines. Two work, these are builds 18363 and 19041. One machine freezes the exact way as described by everyone in this issue, build 19042.

[tomdod]

My machine with Windows build 19043.1237 does not work, but it works fine on a machine running 19042.1266.

[amarhod]

My machine with Windows build 19043.1237 also freezes when writing figures. Running the CMD script yields:

[1014/172325.599:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale
[1014/172325.636:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir. 
{"code": 1, "message": "No Scope Specified", "result": null, "version": "0.2.1"}

[stroitzsch]

I'm seemingly having the same issue on Windows 10 Build 19043.1320.

Has anyone figured out a workaround for this?

[lordee]

I changed back to the other engine (orca?)

[amarhod]

I'm seemingly having the same issue on Windows 10 Build 19043.1320.

Has anyone figured out a workaround for this?

It works with WSL for me

[manujosephv]

I have the exact same issue. downgrading kaleido to 0.1.0 solved it for me.

[michaeloliverx]

I am also having this issue on Windows 10 Build: 19043.1348.

write_image will hang forever and cannot be killed with Ctrl+C.

[StSchulze]

Same issue here, with Windows 11 Version 10.0.22000

I was able to get it to work by downgrading kaleido to 0.1.0post1 (0.1.0 didn't work for me).

[mshakib]

I hope you find this helpful I was able to resolve the issue by downgrade Kaleido to 0.1.0 and Configuring the Kaleido library to see the path for the Kaleido executable. here is the change that I made to the Kaleido library:

file: Kaleido\scope\base.py  Line:70

  @classmethod
  def executable_path(cls):
      vendored_executable_path = os.path.join(
          os.path.dirname(os.path.dirname(os.path.abspath(__file__))),
          'executable',
          'kaleido.cmd'

      )

[jglascock]

I had the same result at @StSchulze ...downgrading to kaleido to 0.1.0post1 worked without any issues.

[koushikam]

I have the same issue on windows 10. Downgrading to 0.1.0 didn't resolve the issue.

[Prajval-1608]

Please note downgrading to 0.1.0 doesn't work. Downgrading to 0.1.0post1 DOES WORK.

[tchannagiri]

Thanks for the tips. I can confirm that kaleido 0.1.0post1 works but not the current latest (0.2.1). I haven't tried 0.1.0.

[mherrmann3]

Same problem here (Windows 10 build 19044.1586). conda-forge doesn't have a 0.1.0post1 , but its provided 0.1.0 solved the issue also for me (provided conda-forge is your preferred channel):

conda install python-kaleido=0.1.0

[j-k-f]

Same problem here, windows: 10.0.19042 Build 19042 Ran install for kaleido 0.1.0post1 in terminal fixed for me as well. pip install kaleido==0.1.0post1

[josetduarte]

Had the same problem, kaleido version 0.1.0post1 fixed the issue

[ThijsvZ]

Windows 10 home, version 21H2, build 19044.1766 VS code 1.67.2. conda 4.13.0 plotly 5.8.2 python 3.10.5 Kaleido 0.2.1

I'm using the MWA provided by @ammaar8, running in a local notebook in VS code. Kaleido 0.2.1. hangs the notebook server. The cell just keeps computing (tested >15mins) and does not respond to an interrupt. Kernel restart is the only solution.

Mathjax

I initially tried disabling mathjax, but that was unsuccessful.

import plotly.io as pio
pio.kaleido.scope.mathjax = None

I now realize that disabling MathJax solves an issue specific to pre-5.0 plotly and pre-0.2 Kaleido, which 0.2.1. resolves by including MathJax.

Downgrade Kaleido

Conda-forge provides version Kaleido 0.1.0post1 under the version label 0.1.0. This version works perfectly.

conda install python-kaleido=0.1.0
pip install kaleido==0.1.0post1

After removing 0.2.1, the installation of 0.1.0 threw an error due to a stale version of the \<env-name>\Libray\bin\KaleidoApp folder. Install worked after a reboot and manual removal of the folder.

[mherrmann3]

Does anyone have an idea why this happens? Here one reason why this downgrade may be undesired:

I noticed that v0.1.0(post1) and v0.2.1 do not produce the exact same PDF files because the fonts are slightly different:

• v0.1.0 uses Postscript Type 3 font
• v0.2.1 uses OpenSans (as does plotly by default)

Strangely, the fonts in the PNGs do not differ that much, they're almost identical. I would judge the PDFs of v0.2.1 to be more similar to the PNGs.

I guess this font difference is related to the upgrade of the Chromium library in v0.2.0, because the Skia PDF versions are different (in the PDF Producer field):

• v0.1.0: Skia/PDF m83
• v0.2.0 Skia/PDF m91

Hence, the proposed solution of downgrading Kaleido is not beneficial for reproducibility and consistency b/w PDFs and PNG.

[whaleprince]

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

my output is windows(normal): D:\ProgramData\Anaconda3\lib\site-packages\kaleido\executable\kaleido.cmd plotly --plotlyjs='D:\ProgramData\Anaconda3\lib\site-packages\plotly\package_data\plotly.min.js' --mathjax='file:///D:/ProgramData/Anaconda3/lib/site-packages/kaleido/executable/etc/mathjax/MathJax.js' --disable-gpu --allow-file-access-from-files --disable-breakpad --disable-dev-shm-usage --no-sandbox

linux(problem): /home/python/software/Python-3.8.8/lib/python3.8/site-packages/kaleido/executable/kaleido plotly --plotlyjs='/home/python/software/Python-3.8.8/lib/python3.8/site-packages/plotly/package_data/plotly.min.js' --mathjax='file:///home/python/software/Python-3.8.8/lib/python3.8/site-packages/kaleido/executable/etc/mathjax/MathJax.js' --disable-gpu --allow-file-access-from-files --disable-breakpad --disable-dev-shm-usage --no-sandbox

[ronak1009]

I am also facing the same issue on Windows 10 and centos7 with Python 3.9

I am not able to run the basic example properly. It takes very very long time ~ 2-3 mins to completely run the following example from the plotly doc.

import plotly.express as px

df = px.data.gapminder().query("country=='Canada'")
fig = px.line(df, x="year", y="lifeExp", title='Life expectancy in Canada')

If I try to write the image, it would again take more than 40 secs to run. fig.write_image("fig1.png", engine='kaleido')

import plotly.io as pio
print(' '.join(pio.kaleido.scope._build_proc_args()))

Output for the following command to get the full command of the kaleido executable is:

C:\Python\Python39\lib\site-packages\kaleido\executable\kaleido.cmd plotly --plotlyjs='C:\\Python\\Python39\\lib\\site-packages\\plotly\\package_data\\plotly.min.js' --mathjax='file:///C:/Python/Python39/lib/site-packages/kaleido/executable/etc/mathjax/MathJax.js' --allow-file-access-from-files --disable-breakpad --disable-dev-shm-usage --no-
sandbox --disable-webgl

And console output to this is:

[1118/165643.734:WARNING:resource_bundle.cc(405)] locale_file_path.empty() for locale
[1118/165643.761:WARNING:headless_browser_main_parts.cc(83)] Cannot create Pref Service with no user data dir.
{"code": 0, "message": "Success", "result": null, "version": "0.2.1"}
[1118/165643.817:ERROR:gpu_init.cc(430)] Passthrough is not supported, GL is swiftshader
[1118/165643.832:ERROR:command_buffer_proxy_impl.cc(121)] ContextResult::kTransientFailure: Failed to send GpuChannelMsg_CreateCommandBuffer.

Note: I could not paste JSON mentioned above.

My env: Python 3.9.13 plotly==5.10.0 kaleido==0.2.1

[Hoernchen]

Same issue here, this can easily be fixed by downgrading kaleido pip install --upgrade "kaleido==0.1.*"

• the problem is that this is not obvious, plotly itself is a dependency of the package I am actually using, and some weird unmaintained kaleido backend thing getting stuck is.. unexpected.

[michaeloliverx]

Anyone using Poetry might find this useful, only installs the old version if the platform is Windows:

kaleido = [
    # Newest version hangs on Windows, see https://github.com/plotly/Kaleido/issues/110
    {version = "0.1.0post1", markers = "platform_system == 'Windows'" }, 
    {version = "0.2.1", markers = "platform_system != 'Windows'"}
]

[stanos4]

any other work-arounds besides downgrading to kaleido 0.1.0post1? I'm on w11 machine, 22000.1455, have the same issue. Problem with downgrade is that you're unable to export images of large plots - i'm particularly speaking about ~30 million points heatmap, currently with many NaN points.

https://github.com/plotly/Kaleido/issues/19

[AmedBrook]

same issue , downgrading to kaleido=0.1.0 works for me on windows 11

[TheNoneMan]

Same issue, Windows 10 19044.2846 While investigating and following this thread discovered that CMD immediately closes for my particular user, meanwhile I was working from powershell just fine. Managed to resolve this by deleting registry entry

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Command Processor

Afterward, CMD started launching fine as well as Kaleido subprocess. I assume it is because Kaleido is being launched via Subprocess library with shell=True and its readiness is assumed based on console output, specifically json line

{"code: 0,...}

And since CMD wasn't launching normally, there wasn't any output, any json to assume Kaleido readiness. Edit: By the end I stuck to using plotly=5.14.1 and kaledio=0.2.1

[hhaensel]

Retried to reproduce the error today but the issue seems to have disappeared on my updated Windows 11 and Windows 10 machines.

[amks1]

I got the same issue in Windows 11.

[MaartenBW]

I still have this issue on Windows 11 with kaleido 0.2.1.

0.2.1.post1 is for some reason niet available for my platform (windows 11, 64-bit processor, python 3.11)

[szmcdull]

Sad for such a long time no one is fixing this

[neuropathbasel]

After about 2 years of encountering this issue (plotly stalls or gets an error when writing plots to PNG on some computers I use, but not all) I found a solution. This does not seem to be linked to particular kaleido/plotly version combinations but essentially boils down to headless (or sort of headless) systems.

kaleido (based on chromium, as I understood) can be launched with command line arguments that make it ignore GPU support, physical or xsever screens etc. However, these command line options need to be passed to kaleido from plotly. Here is how (e.g., WLS, Windows 10, Ubuntu 20.04, python 3.7, kaleido==0.2.1, plotly==5.3.1):

I got the idea here: https://discuss.streamlit.io/t/unable-to-run-plotly-kaleido-on-streamlit-cloud-to-generate-image-exports-from-graphs/23425/6

You can test this in Jupyer notebook:

import kaleido
import plotly.express as px
import plotly.io as pio

# set explicit headless parameters for chromium (not sure if all are needed)
pio.kaleido.scope.chromium_args = (
    "--headless",
#    "--no-sandbox",
    "--single-process",
    "--disable-gpu"
)  # tuple with chromium args

fig = px.scatter(x=[0, 1, 2, 3, 4], y=[0, 1, 4, 9, 16])
fig.show()
fig.write_html('first_figure.html', auto_open=False)
fig.write_image('first_figure.png') # fails with kaleido 0.1.0, 0.2.1

# https://github.com/plotly/Kaleido/issues/134

Here's what happens behind the scenes:

1. plotly calls kaleido
2. kaleido binary starts, headless paremeters not set
3. kaleido crashes
4. plotly waits indefinitely for kaleido to respond, which will never happen (like here https://github.com/plotly/Kaleido/issues/134)

If you launch the kaleido binary (e.g. from ... /lib/python/site-packages.... /kaleido on a headless system (like WSL, which is formally headless) , you will see a crash:

3f)
  r8: 0000000000000000  r9: 00007fffec0c87c0 r10: 0000000000000008 r11: 0000000000000008
 r12: 00007fffec0c9260 r13: 00007fffec0c9ad0 r14: 00007fffec0c9ae0 r15: 00007fffec0c9ae8
  di: 0000000000000002  si: 00007fffec0c87c0  bp: 00007fffec0c8a10  bx: 00007f0f7eee0e00
  dx: 0000000000000000  ax: 0000000000000000  cx: 0000000000000008  sp: 00007fffec0c87c0
  ip: 00007f0f7f36300b efl: 0000000000000246 cgf: 00000053002b0033 erf: 0000000000000000
 trp: 0000000000000000 msk: 0000000000000000 cr2: 0000000000000000
[end of stack trace]
Calling _exit(1). Core file will not be generated.

If you launch kaleido on the same headless system with above-mentioned command line arguments, it runs perfectly fine:

 ./kaleido --headless --no-sandbox --single-process --disable-gpu
[0515/220312.534508:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
[0515/220312.616704:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
[0515/220312.652037:WARNING:resource_bundle.cc(431)] locale_file_path.empty() for locale
{"code": 1, "message": "No Scope Specified", "result": null, "version": "0.2.1"}

I'm not sure that there is a simple explanation why some version combinations of plotly and kaleido fixed the issue. I tried all of these suggestions (without success) before I encountered above-mentioned fix. And there is a logical explanation to it.

[neuropathbasel]

I outcommented the --no-sandbox option. It can lead up to orphaned kaleido processes as I just found out. These keep running after completing their task. And the option is not required to make it work.

[ant1j]

@neuropathbasel Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3.11.7, kaleido==0.2.1, but I got an error message (better than hanging forever though)

[neuropathbasel]

I have no Windows machine where I could try. Its an institutional Windows system with admin rights only within WSL. I have never used Python in Windows before.

-- Dr. med. Jürgen Hench

Oberarzt, Neuropathologie Universitätsspital Basel | Institut für Medizinische Genetik und Pathologie Schönbeinstrasse 40 | CH-4031 Basel

Tel +41 61 328 6891 | Fax +41 61 265 3194 E-Mail: @.*** http://pathologie.unispital-basel.ch

ü Please consider our environment before printing out this e-mail. Bitte denken Sie an unsere Umwelt, bevor Sie dieses E-Mail ausdrucken!

---

Von: Antoine Jouanjean @.***> Gesendet: Dienstag, 21. Mai 2024 11:17:58 An: plotly/Kaleido Cc: Hench Juergen, Dr.; Mention Betreff: Re: [plotly/Kaleido] Image generation hangs on windows 10 (#110)

@neuropathbasel Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3. 11. 7, kaleido==0. 2. 1, but I got an error message (better than hanging forever though) — Reply to this email directly, view it on ZjQcmQRYFpfptBannerStart Diese Nachricht stammt von einem externen Absender Diese Nachricht kam von ausserhalb Ihrer Organisation.

ZjQcmQRYFpfptBannerEnd

@neuropathbaselhttps://urldefense.com/v3/__https://github.com/neuropathbasel__;!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXxxOFcAI$ Do you confirm it works for WSL but not Windows 11 natively? Just tried on W11, Python 3.11.7, kaleido==0.2.1, but I got an error message (better than hanging forever though)

— Reply to this email directly, view it on GitHubhttps://urldefense.com/v3/__https://github.com/plotly/Kaleido/issues/110*issuecomment-2122162630__;Iw!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXULKsNho$, or unsubscribehttps://urldefense.com/v3/__https://github.com/notifications/unsubscribe-auth/AOIQ23LB6MPNM7GTXEHSFOLZDMGMNAVCNFSM5DMFOE62U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJSGIYTMMRWGMYA__;!!EDSXHx-qqdzzoNk!tv6aMNnX5GLqCQXXaEjCwHY4Hxnvkrfiw7OHGAABfmdG88DFef1mYcimv6diVP7NKFFDdFP-XPsD5NXF_sZXgjyvtdU$. You are receiving this because you were mentioned.Message ID: @.***>