Closed andreped closed 1 year ago
Alternatively, you could try pyFAST by going into the built docker container:
docker run -it --rm h2gnet:latest
and then running:
python3 -c "import fast"
which in my case results in:
WARNING [139799543730176] Unable to open X display. Disabling visualization.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/fast/__init__.py", line 22, in <module>
fast.ImageFileImporter.create('') # Trigger splash, GL context initialization etc.
File "/usr/local/lib/python3.10/dist-packages/fast/fast.py", line 13671, in create
return _fast.ImageFileImporter_create(filename)
RuntimeError: clGetPlatformIDs
I then installed clinfo
by:
apt-get install clinfo
from inside the docker container, and then run clinfo
I got:
root@9d86e9f491eb:/# clinfo
Number of platforms 0
If I then try to install:
apt install mesa-opencl-icd
I get that there is one platform. Trying pyFAST again it prompts:
WARNING [140231796592640] Unable to open X display. Disabling visualization.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/local/lib/python3.10/dist-packages/fast/__init__.py", line 22, in <module>
fast.ImageFileImporter.create('') # Trigger splash, GL context initialization etc.
File "/usr/local/lib/python3.10/dist-packages/fast/fast.py", line 13671, in create
return _fast.ImageFileImporter_create(filename)
RuntimeError: No valid OpenCL platforms!
Have you tried the Nvidia docker and installed Nvidia ooencl ICD?
Have you tried the Nvidia docker and installed Nvidia ooencl ICD?
Nvidia has a lot of different docker images. Do you mean this one nvidia/opencl. If so, yes, I have, same issue.
Regarding the nvidia-opencl-icd
, I was unable to find it directly, but seems like I can install it if I choose a driver. I can give that a go.
Otherwise, there is the nvidia/opengl image, which is a lot newer than the OpenCL docker. I can try that next. But feels like I'm just doing all permutations now. If you were to set this up on a fresh ubuntu machine (without NVIDIA GPU) which docker image would you use and what would you install in addition (if necessary)?
Made a new Dockerfile according to the linuxhandbook you suggested above:
# build custom image from the ubuntu docker image
FROM ubuntu:20.04
ARG DEBIAN_FRONTEND=noninteractive
# maintainer of docker file
# MAINTAINER André Pedersen <andre.pedersen@sintef.no>
# set language, format, and stuff
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
# install dependencies and OpenCL
RUN apt-get update && apt-get -y upgrade \
&& apt-get install -y \
apt-utils \
unzip \
tar \
curl \
xz-utils \
ocl-icd-libopencl1 \
opencl-headers \
clinfo \
python3-pip \
;
RUN mkdir -p /etc/OpenCL/vendors && \
echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
# Install OpenGL
RUN apt-get install -y \
libopengl0 \
libgl1 \
libusb-1.0-0-dev \
libcairo2
# install python3 dependencies
RUN pip3 install --default-timeout=1000 pyfast==4.6.0
Made minor adjustments to add OpenGL support and to install pyFAST. To reproduce do:
git clone https://github.com/andreped/H2G-Net.git
cd H2G-Net/src/docker/
docker build -t h2gnet .
docker run -it --rm h2gnet:latest runPipeline --verbose
Which results in the same issue as earlier (even on Ubuntu 18.04):
INFO [140383789644544] Loaded configuration file: /usr/local/lib/python3.8/dist-packages/fast/bin/../bin/fast_configuration.txt
INFO [140383789644544] Test data path: /root/FAST/data/
INFO [140383789644544] Kernel source path: /usr/local/lib/python3.8/dist-packages/fast/bin/../bin//..//kernels/
INFO [140383789644544] Kernel binary path: /root/FAST//kernel_binaries/
INFO [140383789644544] Documentation path: /usr/local/lib/python3.8/dist-packages/fast/bin/../bin//..//doc/
INFO [140383789644544] Pipeline path: /root/FAST//pipelines/
INFO [140383789644544] Qt plugins path: /usr/local/lib/python3.8/dist-packages/fast/bin/../bin//..//plugins/
INFO [140383789644544] Library path: /usr/local/lib/python3.8/dist-packages/fast/bin/../bin//..//lib/
INFO [140383789644544] Creating new QApp
WARNING [140383789644544] Unable to open X display. Disabling visualization.
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
INFO [140383789644544] QApp already exists..
ERROR [140383789644544] Terminated with unhandled exception: clGetPlatformIDs
Hence, there is still something that is not working. Perhaps the docker image is not able to access the resources from the local hardware. Not sure. Any ideas what I could try?
I manged to fix the issue by building the docker image from the NVIDIA CUDA image, where I set the CUDA version to match what I have where the docker image will be used.
In the end, this is the Dockerfile that worked with pyFAST:
# build custom docker image from the nvidia CUDA image
FROM nvidia/cuda:11.0.3-cudnn8-runtime-ubuntu20.04
# headless
ARG DEBIAN_FRONTEND=noninteractive
# set language, format, and stuff
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
# install dependencies and OpenCL
RUN apt-get update && apt-get -y upgrade \
&& apt-get install -y \
apt-utils \
unzip \
tar \
curl \
xz-utils \
ocl-icd-libopencl1 \
opencl-headers \
clinfo \
python3-pip \
;
RUN mkdir -p /etc/OpenCL/vendors && \
echo "libnvidia-opencl.so.1" > /etc/OpenCL/vendors/nvidia.icd
ENV NVIDIA_VISIBLE_DEVICES all
ENV NVIDIA_DRIVER_CAPABILITIES compute,utility
# Install OpenGL
RUN apt-get install -y \
libopengl0 \
libgl1 \
libusb-1.0-0-dev \
libcairo2
# install python3 dependencies
RUN pip3 install --default-timeout=1000 pyfast==4.6.0
As this issue has been resolved, I'm closing this issue.
However, it could be a good idea to update the documentations to include how to use FAST with Docker, @smistad.
Describe the bug I was trying to use FAST within a docker container, but it seems to fail due to some OpenCL issues, likely installation gone wrong or runPipeline requiring there to be a display to render the viewer.
To Reproduce Here is the extremely simple Dockerfile I made:
First create the Dockerfile above and build the docker image (from where the Dockerfile lies):
Then simply check if
runPipeline
can be called within the docker image by:System:
Expected behavior Should be able to use runPipeline on a headless machine, such as within docker. Even a machine without GPU (only Intel CPU for instance). Same for
pyFAST
which appear to have the same limitation.Error log Produced from running the final command above: