Kaleido subprocess failed on Ubuntu 20.04.2 LTS x86_64

[meassinal]

In my python dash application I tried to export figure image fig_img = figure.to_image(format="png", engine="kaleido") and then post the image to s3. But, there was an error with kaleido on my Ubuntu 20.04.2 LTS x86_64 server as the following:

response = self._perform_transform(
  File "/home/ubuntu/.virtualenvs/py39cv45/lib/python3.9/site-packages/kaleido/scopes/base.py", line 293, in _perform_transform
    self._ensure_kaleido()
  File "/home/ubuntu/.virtualenvs/py39cv45/lib/python3.9/site-packages/kaleido/scopes/base.py", line 198, in _ensure_kaleido
    raise ValueError(message)
ValueError: Failed to start Kaleido subprocess. Error stream:

/home/ubuntu/.virtualenvs/py39cv45/lib/python3.9/site-packages/kaleido/executable/kaleido: line 2: dirname: command not found
/home/ubuntu/.virtualenvs/py39cv45/lib/python3.9/site-packages/kaleido/executable/kaleido: line 10: ./bin/kaleido: No such file or directory

On my Ubuntu server, I installed kaleido version 3 and I'm using python 3.9. Unlike Ubuntu, it worked just fine on my macOS as a development machine. Could anyone help point out what I should configure to make it work?

[jonmmease]

Hi @meassinal, thanks for the report. From the error message, it looks like there might be an issue with how the PATH environment variable is configured. dirname is a built-in command that's usually located somewhere like /usr/bin/dirname, and the /usr/bin directory is expect to be listed in the PATH environment variable.

This can happen if there is a config file somewhere that accidentally overwrites the PATH, instead of adding to it. For example, doing export PATH=/my/dir instead of export PATH=$PATH:/my/dir.

From your python script, you can run the following command to print out all of the directories that are on your path:

import os
print(os.environ["PATH"].replace(":", "\n"))

Also, kaleido 0.0.3 is a pretty old version at this point. Are you able to update? (current latest version of 0.2.1).

[meassinal]

Hi @jonmmease, I am so grateful for your response. Following your points, I could figure out what might have caused the problem. In my supervisor config file, I did have environment=PATH=/home/ubuntu/.virtualenvs/py39cv45/bin/python3.9 . But, after I tried updating to environment=PATH=$PATH:/python3.9 it still shows the same issue. Even though, I removed environment from the supervisor, the error remains.

Additionally, here are relevant directories on the path by python script:

/home/ubuntu/.virtualenvs/py39cv45/bin
/usr/local/sbin
/usr/local/bin
/usr/sbin
/usr/bin
/sbin
/bin
/usr/games
/usr/local/games
/snap/bin

where /usr/bin exists in the path, yet dirname called inside kaleido bash script couldn't find it somehow.

I've already updated my kaleido to current latest version of 0.2.1. Could you help to point out the issue?

[jonmmease]

Hi @meassinal,

Here are a couple of things that might be relevant.

• https://serverfault.com/questions/331027/supervisord-how-to-append-to-path
• https://github.com/Supervisor/supervisor/blob/master/supervisor/skel/sample.conf

I'm not very familiar with supervisord, but it looks like the syntax for referencing the existing PATH environment variable is not the same as it is for Bash.

Maybe try something like this

environment=PATH="/other/path/:%(ENV_PATH)s"

Also, just to double check, have you confirmed that you do have a dirname executable in /usr/bin?

[meassinal]

Hi @jonmmease, I've tried environment=PATH="/home/ubuntu/.virtualenvs/py39cv45/bin:/usr/bin:%(ENV_PATH)s" for supervisor config yet still the error remains. Fortunately, however, I happened to modify kaleido bash script by using full path of dirname /usr/bin/dirname and then it worked 😄 Here is modified version of kaleido bash script on my ubuntu 20.04.2 LTS x86_64 server which lies in the path: /home/ubuntu/.virtualenvs/py39cv45/lib/python3.9/site-packages/kaleido/executable/kaleido

#!/bin/bash
DIR="$( cd "$( /usr/bin/dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export FONTCONFIG_PATH=$DIR/etc/fonts
export XDG_DATA_HOME=$DIR/xdg
unset LD_PRELOAD

cd $DIR
./bin/kaleido $@

I'd like to thank you so much for relevant key points to the error that without them I couldn't figure it out. I really appreciate your effort 🙏

[jonmmease]

Thanks for the update @meassinal, though it is still a bit troubling.

As one last experiment, could you try adding an echo $PATH to the Kaleido script to see what is on the PATH by the time the script runs? I expect this would be added to a supervisord log file.

I'm wondering if we should consider searching for dirname in some common directories if it's not on the path :thinking:

[meassinal]

Hi @jonmmease , The last experiment turned out that the path in supervisor process contained only my python path environment like so /home/ubuntu/.virtualenvs/py39cv45/bin/python3.9.

Here how I produced the result:

#!/bin/bash

echo "$PATH" > "/tmp/test_path.txt"

DIR="$( cd "$( /usr/bin/dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export FONTCONFIG_PATH=$DIR/etc/fonts
export XDG_DATA_HOME=$DIR/xdg
unset LD_PRELOAD

cd $DIR
./bin/kaleido $@

and when cat /tmp/test_path.txt the output is /home/ubuntu/.virtualenvs/py39cv45/bin/python3.9 so that it's the reason dirname could not be found via system path.

Probably, in some special cases you should use full executable path of /usr/bin/dirname.

Cheers!

[DustinCai]

Hi @jonmmease , The last experiment turned out that the path in supervisor process contained only my python path environment like so /home/ubuntu/.virtualenvs/py39cv45/bin/python3.9.

Here how I produced the result:

#!/bin/bash

echo "$PATH" > "/tmp/test_path.txt"

DIR="$( cd "$( /usr/bin/dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

export LD_LIBRARY_PATH=$DIR/lib:$LD_LIBRARY_PATH
export FONTCONFIG_PATH=$DIR/etc/fonts
export XDG_DATA_HOME=$DIR/xdg
unset LD_PRELOAD

cd $DIR
./bin/kaleido $@

and when cat /tmp/test_path.txt the output is /home/ubuntu/.virtualenvs/py39cv45/bin/python3.9 so that it's the reason dirname could not be found via system path.

Probably, in some special cases you should use full executable path of /usr/bin/dirname.

Cheers!

Were you able to find a more elegant solution for this instead of replacing the line in the file?