Closed WurtzJd closed 3 years ago
Hi @WurtzJd,
Inside a docker container you'll probably need to run orca with xvfb-run. See https://github.com/plotly/orca/blob/80855093193afba09a4dbebaa7bae776be88e15e/README.md#linux-troubleshooting-headless-server-configuration in the README. Also, in case it's useful to you, here's the docker configuration used by the plotly team uses https://github.com/plotly/orca/blob/master/deployment/Dockerfile
@WurtzJd did using xvfb-run
like @jonmmease suggested solve your problem?
@WurtzJd would you see a lot of value in us publishing an official Docker image for Orca?
In the meantime, you can find an example Dockerfile there: https://github.com/plotly/orca/blob/master/deployment/Dockerfile
@jonmmease , I struggle to understand how to use the shell script. Where is supposed to be the "orca-X.Y.Z-x86_64.App" file? The docker config you sent is a bit complex and I don't see which part I should use.
@antoinerg, it would be very useful to have a simple docker image indeed
I also fail to use plotly orca outside docker when ssh into another machine. pio.write_image returns:
When used on Linux, orca requires an X11 display server, but none was detected. Please install X11, or configure your system with Xvfb
I just tried to install conda again, following the Git README: 1) download orca-1.2.1-x86_64.AppImage 2) chmod +x orca-1.2.1-x86_64.AppImage 3) ln -s /home/me/orca-1.2.1-x86_64.AppImage /home/me/.pyenv/shims/orca
Now when I try orca --help I get
/home/me/.pyenv/shims/orca: 1: /home/me/.pyenv/shims/orca: Syntax error: redirection unexpected
And xvfb-run -a /home/me/orca-1.2.1-x86_64.AppImage "$@"
returns /home/jd/orca-1.2.1-x86_64.AppImage: 1: /home/jd/orca-1.2.1-x86_64.AppImage: Syntax error: redirection unexpected
Trying to get orca installed on an ubuntu docker image is a nightmare! Has anyone been able to do it?
@scottwilson312 I've been on two sprints to try and do this in a docker image from Jupyter for use in a MyBinder session, and share your frustration.
Dockerfile for repo: https://github.com/electropy/notebooks/blob/orca/Dockerfile MyBinder link: https://mybinder.org/v2/gh/electropy/notebooks/orca?urlpath=lab
At this point, I'm getting two issues.
In a terminal:
jovyan@jupyter-electropy-2dnotebooks-2dwlf11omd:~/work/notebooks$ orca --help
fuse: device not found, try 'modprobe fuse' first
Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
Cannot mount AppImage, please check your FUSE setup.
You might still be able to extract the contents of this AppImage
if you run it with the --appimage-extract option.
See https://github.com/AppImage/AppImageKit/wiki/FUSE
for more information
open dir error: No such file or directory
And trying to use it in a Juptyer Notebook regardless, seems like it doesn't see orca in the python/jupyter path:
import plotly.io as pio pio.write_image(fig, 'fig1.png')
ValueError:
The orca executable is required in order to export figures as static images,
but the executable that was found at '/home/jovyan/work/notebooks/bin/orca'
does not seem to be a valid plotly orca executable. Please refer to the end of
this message for details on what went wrong.
If you haven't installed orca yet, you can do so using conda as follows:
$ conda install -c plotly plotly-orca
Alternatively, see other installation methods in the orca project README at
https://github.com/plotly/orca.
After installation is complete, no further configuration should be needed.
If you have installed orca, then for some reason plotly.py was unable to
locate it. In this case, set the `plotly.io.orca.config.executable`
property to the full path of your orca executable. For example:
>>> plotly.io.orca.config.executable = '/path/to/orca'
After updating this executable property, try the export operation again.
If it is successful then you may want to save this configuration so that it
will be applied automatically in future sessions. You can do this as follows:
>>> plotly.io.orca.config.save()
If you're still having trouble, feel free to ask for help on the forums at
https://community.plot.ly/c/api/python
Here is the error that was returned by the command
$ /home/jovyan/work/notebooks/bin/orca --help
[Return code: 1]
Note: When used on Linux, orca requires an X11 display server, but none was
detected. Please install X11, or configure your system with Xvfb. See
the orca README (https://github.com/plotly/orca) for instructions on using
orca with Xvfb.
This has been so painful and tiring, but I really really really want this feature to work :/
I've tried other variants of installations (e.g. from conda), but they didn't work because MyBinder sessions are headless, I think.
To solve the Fuse error to try running docker as privileged
On Fri, Apr 12, 2019, 7:02 AM Mathieu Boudreau notifications@github.com wrote:
I've tried other variants of installations (e.g. from conda), but they didn't work because MyBinder sessions are headless, I think.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/plotly/orca/issues/150#issuecomment-482585956, or mute the thread https://github.com/notifications/unsubscribe-auth/APty_nrAwIeAxOxtCGcKZbXudb526xVZks5vgJHcgaJpZM4Yz1YV .
Unfortunately, I don't think I have control over that as the MyBinder service is what's running the docker image after it's built; I don't set the commands myself. Thanks though!
Hi @mathieuboudreau,
From your error messages, it looks like core issue is that your docker container is unable to use fuse to mount the AppImage. The AppImage format is really an archive containing a potentially complex file structure of the program and all the dependencies. When you run an AppImage, the image file is automatically mounted using fuse in an unarchived form so that the executable inside the archive can be run like a normal linux program.
I'm not experienced enough with using fuse inside docker to know if there is an easy way to get fuse working, so I'd recommend trying the --appimage-extract
optioned mentioned in the error message (and described in the AppImage documentation at https://github.com/AppImage/AppImageKit/wiki/FUSE#docker).
Something like
# Download orca AppImage, extract it, and make it executable under xvfb
RUN wget https://github.com/plotly/orca/releases/download/v1.1.1/orca-1.1.1-x86_64.AppImage -P /home
RUN chmod 777 /home/orca-1.1.1-x86_64.AppImage
# To avoid the need for FUSE, extract the AppImage into a directory (name squashfs-root by default)
RUN cd /home && /home/orca-1.1.1-x86_64.AppImage --appimage-extract
RUN printf '#!/bin/bash \nxvfb-run --auto-servernum --server-args "-screen 0 640x480x24" /home/squashfs-root/app/orca "$@"' > /usr/bin/orca
RUN chmod 777 /usr/bin/orca
Hope that helps!
Thanks for that tip @jonmmease !
Unfortunately, now I get a [Return code: 1]
. I think at the end of the day, MyBinder's BinderHub settings don't allow for extending the docker image permissions to --privileged
that are needed to run the orca server, so that might be the final wall that I can't jump. I'll just have to stick with a local setup, which isn't the end of the world, it just would have been simpler for my users if I could have this all contained in the web for them.
When you run $ orca --help
from the command line inside you're container you're getting [Return code: 1]
? It's not clear to be what (besides potentially FUSE) would be requiring privileged permissions. 🤔
jovyan@jupyter-electropy-2dnotebooks-2drrm4ffxs:~/work/notebooks$ orca --help
/usr/bin/xvfb-run: 184: /usr/bin/xvfb-run: /home/squashfs-root/app/orca: Permission denied
jovyan@jupyter-electropy-2dnotebooks-2drrm4ffxs:~/work/notebooks$
I tried to RUN chmod 777 /home/squashfs-root/app/orca
in my docker file here, but it still throws the same error as above with it.
Over in https://github.com/plotly/plotly.py/pull/1523 I started working on an update to plotly.py to automatically detect whether it should try to use Xvfb when calling orca. Basically, if we're on Linux, there's no X11 server active (based on the lack of a DISPLAY
environment variable), and xvfb-run
is on the path then use xvfb automatically.
With this branch, I was able to get the orca conda package working in Colab (https://github.com/plotly/plotly.py/pull/1523#issuecomment-483460839).
I'm wondering if this approach would work on binder since the conda package isn't using the AppImage format.
And one idea with your current approach. I opened up the binder link and opened a terminal and ran:
jovyan@jupyter-electropy-2dnotebooks-2d9eojis5z:~/work/notebooks$ ls -la /home
total 48024
drwxr-xr-x 1 root root 4096 Apr 15 13:22 .
drwxr-xr-x 1 root root 4096 Apr 16 00:04 ..
drwsrwsr-x 1 jovyan users 4096 Apr 16 00:05 jovyan
-rwxrwxrwx 1 root root 49151689 Aug 30 2018 orca-1.1.1-x86_64.AppImage
drwx------ 1 root root 4096 Apr 15 13:22 squashfs-root
Based on this, it looks like the permissions you set on the AppImage file were set properly. I think the problem here might be that the top-level squashfs-root directory is still locked down to root
even though you opened up permissions on the orca executable inside with chmod 777 /home/squashfs-root/app/orca
.
Try replacing
RUN chmod 777 /home/squashfs-root/app/orca
with
RUN chmod -R 777 /home/squashfs-root/
to recursively open up the permission of the entire directory structure.
AMAZING! It finally works! Thanks so much @jonmmease !!
I needed one more apt-get install for it to work using my base notebook (libxss1
), but now it works perfectly; I can export static plotly images using orca in a MyBinder session.
I stripped my notebook to the bare essentials (more or less), and uploaded it in a separate repo along with an example Jupyter Notebook from Plotly's own "Static image export in python" example.
Dockerfile: https://github.com/mathieuboudreau/orca-plotly-dockerfile/blob/master/Dockerfile Repo: https://github.com/mathieuboudreau/orca-plotly-dockerfile MyBinder: https://mybinder.org/v2/gh/mathieuboudreau/orca-plotly-dockerfile/master
That's great @mathieuboudreau, and thanks for making that repo to help other folks work through this in the future!
Awesome @mathieuboudreau! Do you think it would also work outside the Jupyter? I mean by simply running a python script
The base docker image that my dockerfile pulls installs Ubuntu (see here: https://hub.docker.com/r/jupyter/base-notebook/dockerfile), so maybe try what I did?
When I run the image, Jupyter is launched automatically, any way to stop this from happening? I tried to start the docker image with "FROM ubuntu:18.04" instead of Jupyter but then I run into errors
@WurtzJd I've only ever used Dockerfiles with MyBinder, which opens Jupyter.
I tried to start the docker image with "FROM ubuntu:18.04" instead of Jupyter but then I run into errors
You probable are missing a lot of the additional settings needed (see here: https://hub.docker.com/r/jupyter/base-notebook/dockerfile)
What you could maybe do is instead copy the jupyter base notebook file:
https://hub.docker.com/r/jupyter/base-notebook/dockerfile
Remove the "Configure container startup" lines from it:
# Configure container startup
ENTRYPOINT ["tini", "-g", "--"]
CMD ["start-notebook.sh"]
Then copy-past my own Dockerfile at the end (minus the FROM... at the top).
https://github.com/mathieuboudreau/orca-plotly-dockerfile/blob/master/Dockerfile
If I were to guess, this would have all the same setup as me, but without opening jupyter.
I followed your tips, but instead I just added the line "CMD ["bash"]" at the end of your dockerfile, and it works like a charm.
Running Orca in docker is going to be so useful, many many thanks for working out and sharing the solution!
It is possible to also implement's @mathieuboudreau's solution to using plotly.io / Plotly's orca in MyBinder.org-served binder sessions without the need for a Dockerfile. In the example here I use requirements.txt
for specifying most of the repo-specific dependencies and use postBuild
to handle setting up orca. Additionally, apt.txt
was used for replacing the first section of the Dockefile to install some critical apt packages. Except for the postBuild
file these are just lists of needed packages or dependencies, and so they offer a way to more easily customize a repo than editing than a Dockerfile.
(See here for @betatim's suggestion in this direction.)
Repo: https://github.com/fomightez/orca-plotly-binderized MyBinder: https://mybinder.org/v2/gh/fomightez/orca-plotly-binderized/master?filepath=index.ipynb
Thank you for everyone who blazed the trail here. This was painful. I'm posting my full docker RUN command that performs all necessary steps to make this work, hopefully this will be useful to others. You should be able to copy and paste this into your docker file. This is built on a pretty minimal container with python 3.6 and Ubuntu, so I believe all of the libraries and dependencies you will need are included here. This is the first operation in my Dockerfile.
# Plotly depedencies
RUN apt-get update && \
apt-get install -y --no-install-recommends \
wget \
xvfb \
xauth \
libgtk2.0-0 \
libxtst6 \
libxss1 \
libgconf-2-4 \
libnss3 \
libasound2 && \
mkdir -p /opt/orca && \
cd /opt/orca && \
wget https://github.com/plotly/orca/releases/download/v1.2.1/orca-1.2.1-x86_64.AppImage && \
chmod +x orca-1.2.1-x86_64.AppImage && \
./orca-1.2.1-x86_64.AppImage --appimage-extract && \
rm orca-1.2.1-x86_64.AppImage && \
printf '#!/bin/bash \nxvfb-run --auto-servernum --server-args "-screen 0 640x480x24" /opt/orca/squashfs-root/app/orca "$@"' > /usr/bin/orca && \
chmod +x /usr/bin/orca
Thank you @davidparks21 , this worked!
Thank you all, just wanted to point out that I also needed to install xauth
because of the following error:
xvfb-run: xauth command not found
None of the suggestions here worked for me. Maybe it is because I'm using Ubuntu 16.04 and its libnss3 is not compatible with the chrome version used by Orca? I tried using more recent debs of libnss3 but that didn't work either.
To even reach the libnss3 error I had to follow the instructions of both davidparks21 and jonmmease.
Perhaps libnss3 should be packaged with the AppImage? Regardless, better documentation on running Orca in a docker container would be much appreciated.
Just tested on 18.04, it works.
Thank you all, just wanted to point out that I also needed to install
xauth
because of the following error:xvfb-run: xauth command not found
Thanks @simone-pignotti, I added xauth to the docker RUN command I posted at https://github.com/plotly/orca/issues/150#issuecomment-519192367, it was indeed pre-installed on my container.
@davidparks21 thanks for your dockerfile, i am using R from inside docker
weirdly running orca on bash works but running orca from inside R in the container gives the following error:
"/opt/orca/squashfs-root/app/orca: symbol lookup error: /opt/orca/squashfs-root/app/orca: undefined symbol: _ZN4node20g_upstream_node_modeE\n"
does this ring a bell ? I havve followed different solutions with no avail so far
thank you
@vxg20 unfortunately I am not going to be much help there, I'm running under Python. I never tested anything under R.
I'm having trouble with this step. After extracting from the latest app image version 1.3.1, I do not have this path - squashfs-root/app/orca. I have something called squashfs-root/AppRun as well as squashfs-root/orca and squashfs-root/resources/app/orca. I am not sure where to point usr/bin/orca. Would appreciate some help with this.
Hello @choprashweta, I think you can link it to squashfs-root/orca
.
Note that if you are running as root
, you will probably get an error stating that you must append --no-sandbox
for it to run.
Share a latest version, there is no problem to use in python-slim: 3.6.8
image.
COPY ./orca-1.3.1.AppImage /usr/src/orca.AppImage
WORKDIR /usr/src
RUN apt-get update && \
apt-get install -y --no-install-recommends \
xvfb xauth libgtk-3-0 libxtst6 libxss1 libgconf-2-4 libnss3 libasound2 && \
./orca.AppImage --appimage-extract && rm orca.AppImage && \
printf '#!/bin/bash \nxvfb-run --auto-servernum --server-args "-screen 0 640x480x24 +extension RANDR +extension GLX" /usr/src/squashfs-root/orca "$@" --no-sandbox' > /usr/bin/orca && \
chmod +x /usr/bin/orca
In general, the root user used in docker, so I added the --no-sandbox
parameter by default.
In addition, in the latest python plotly package, it seems to be trying to automatically solve the running problem, such as trying to use nvfb
to start orca
, because I see this paragraph in the error message:
Here is the error that was returned by the command
$ /usr/bin/xvfb-run --auto-servernum --server-args -screen 0 640x480x24 +extension RANDR +extension GLX /usr/bin/orca --help
But this command seems to be wrong. The -screen 0 640x480x24 + extension RANDR + extension GLX
parameter should be included with"
, otherwise it will not work, so the best way is to use the method mentioned above and create an orca
executeable file, which uses xvfb
to callorca
, and can flexibly control the parameters of orca
.
I hope the official can check whether the automatic solution has the parameter problem I mentioned.
I successfully got orca (from R) to run inside a docker with the appImage and using xvfb-run and the --appimage-extract-and-run option. It works for most plotly graphs except for the ones using webGL like the SPLOM graph or scatterGL rendered graphs. anyone have the same problem? anyone knows a solution?
I am using this as /usr/local/bin/orca:
xvfb-run -a --server-args='-screen 0 1024x768x24 +extension GLX +extension RANDR +render' /usr/local/bin/orca-1.3.1.AppImage --appimage-extract-and-run --no-sandbox "$@"
Hello @antoinerg,
I am not running docker image, but on aws EC2 instance (based on anaconda-python3 image). I tried conda plotly-orca
but hit the Cannot open shared object
error, but AWS linux distribution does not have libgtk2.0-0
and libgconf-2-4
packages available. Now trying orca AppImage.
I use --appimage-extract
on app image version 1.3.1
as an ec2-user, and get a squashfs-root
folder under /home/ec2-user. I run:
xvfb-run --auto-servernum --server-args "-screen 0 1280x800x24" /home/ec2-user/squashfs-root/orca "$@" > /opt/conda/envs/pysig/bin/orca
chmod 777 /opt/conda/envs/pysig/bin/orca
However orca --help
returns this:
/opt/conda/envs/pysig/bin/orca: line 1: /home/ec2-user/squashfs-root/orca:: No such file or directory
But /home/ec2-user/squashfs-root/orca
does exist:
ls -l /home/ec2-user/squashfs-root/orca -rwxrwxrwx 1 ec2-user ec2-user 112481216 Jun 29 01:39 /home/ec2-user/squashfs-root/orca
I have installed Xvfb using sudo, run all commands as ec2-user instead of root, b/c on anaconda-python3 image, ec2-user is the owner of all the conda stuff.
Any idea why orca --help
cant find /home/ec2-user/squashfs-root/orca
when it does exist in /home/ec2-user/squashfs-root
folder which I applied a chmod -R 777
already?
I also tried orca-1.2.1-x86_64.AppImage in place of orca-1.3.1.AppImage to no avail.
Orca seems like a big hassle to use, can't find any successful use case examples beyond windows and ubuntu, and no help on here nor stackoverflow nor plotly community.
@davidparks21 I really want to thank you for posting that Dockerfile. I spent most of this morning and afternoon trying to do the exact same thing. Getting orca set up in a Docker container was incredibly painful.
can this be converted to a alpine image? struggling my ass off trying to convert it.
# Plotly depedencies
RUN apt-get update && \
apt-get install -y --no-install-recommends \
wget \
xvfb \
xauth \
libgtk2.0-0 \
libxtst6 \
libxss1 \
libgconf-2-4 \
libnss3 \
libasound2 && \
mkdir -p /opt/orca && \
cd /opt/orca && \
wget https://github.com/plotly/orca/releases/download/v1.2.1/orca-1.2.1-x86_64.AppImage && \
chmod +x orca-1.2.1-x86_64.AppImage && \
./orca-1.2.1-x86_64.AppImage --appimage-extract && \
rm orca-1.2.1-x86_64.AppImage && \
printf '#!/bin/bash \nxvfb-run --auto-servernum --server-args "-screen 0 640x480x24" /opt/orca/squashfs-root/app/orca "$@"' > /usr/bin/orca && \
chmod +x /usr/bin/orca
FYI. A docker image already exists. Just do
docker pull quay.io/plotly/orca
On Sun, 16 Aug 2020 at 14:41, JaredOzzy notifications@github.com wrote:
can this be converted to a alpine image? struggling my ass off trying to convert it.
Plotly depedencies
RUN apt-get update && \ apt-get install -y --no-install-recommends \ wget \ xvfb \ xauth \ libgtk2.0-0 \ libxtst6 \ libxss1 \ libgconf-2-4 \ libnss3 \ libasound2 && \ mkdir -p /opt/orca && \ cd /opt/orca && \ wget https://github.com/plotly/orca/releases/download/v1.2.1/orca-1.2.1-x86_64.AppImage && \ chmod +x orca-1.2.1-x86_64.AppImage && \ ./orca-1.2.1-x86_64.AppImage --appimage-extract && \ rm orca-1.2.1-x86_64.AppImage && \ printf '#!/bin/bash \nxvfb-run --auto-servernum --server-args "-screen 0 640x480x24" /opt/orca/squashfs-root/app/orca "$@"' > /usr/bin/orca && \ chmod +x /usr/bin/orca
— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/plotly/orca/issues/150#issuecomment-674521873, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEVULWVKGZSGDMJOM75KQV3SA7HWVANCNFSM4GGPKYKQ .
Hi there! I made a dash app that has the option to download the plot that is shown, and thus uses orca. Now I'm trying to dockerize the app.
Thanks to @davidparks21 and @459217974 I was able to fix the errors that kept appearing when dealing with orca installation.
The problem is that now I'm trying to follow one Docker security “best practice” of creating a non-root user in the following way:
RUN useradd appuser && chown -R appuser /usr/src/app
RUN chown -R appuser /opt/orca
USER appuser
But when I try to run "orca --help" in the container shell I get the following:
A JavaScript error occurred in the main process
Uncaught Exception:
Error: Failed to get 'appData' path
at App.app._setDefaultAppPaths (/opt/orca/squashfs-root/resources/electron.asar/browser/api/app.js:54:43)
at Object.<anonymous> (/opt/orca/squashfs-root/resources/electron.asar/browser/init.js:133:5)
at Module._compile (internal/modules/cjs/loader.js:786:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:798:10)
at Module.load (internal/modules/cjs/loader.js:645:32)
at Function.Module._load (internal/modules/cjs/loader.js:560:12)
at Function.Module.runMain (internal/modules/cjs/loader.js:850:10)
at internal/main/run_main_module.js:17:11
Does anyone faced the same problem? Thanks in advance!
I'll just mention that we're basically replacing Orca with the new Kaleido project (https://github.com/plotly/Kaleido) which is significantly easier to install in docker containers and is our new recommended way forward for this kind of thing :)
Oh! I wasn't aware of that! It seems much much easier. Thank you very much for the tip @nicolaskruchten! I'll try it right way.
EDIT: It worked like a charm! Really amazing after all the strugling with orca. Thanks again @nicolaskruchten for pointing it out!
@nicolaskruchten Kaleido is great, and very easy to use here after spending way too much time wrangling orca. Thanks for the rec!
Orca can be used under Docker. See example Dockerfile as well as already built images!
thanks @nicolaskruchten; all I needed was
pip install kaleido
and not to use orca
at all, despite what the to_image
error message said. Can that error msg be changed? I tried many things from this thread before realizing it was all unnecessary.
This is pretty much an artifact of the order in which things were released... if you don't have orca
installed you don't get that error message :)
alright, not sure I'm following though…
seems better for the error msg to mention kaleido, or [orca || kaleido]?
Ah I see, indeed the way the defaults work right now, without kaleido
installed the engine is assumed to be orca
and then you get the orca-related error messages, including the "hey you should install orca
" ones, which is probably leading to slower kaleido
adoption than we'd like. This is probably something we should look at changing in v5 @jonmmease ?
Thanks for following up @ryan-williams :)
Yeah, I think I'd actually be in favor of dropping Orca support in v5. AFAIK, we haven't come across any reports from users who are using orca successfully but are unable to use Kaleido.
When using plotly.io.to_image inside a docker container I get : "When used on Linux, orca requires an X11 display server, but none was detected. Please install X11, or configure your system with Xvfb."
I tried adding apt-get install -y x11-apps but it does not help.
Cheers