Closed ArXen42 closed 3 years ago
@shimat Thanks for the dockerfile you mentioned in #953. I modified this file to create base image that builds environment for running apps that use OpenCVSharp
with all necessary dependencies. Modified dockerfile uses Ubuntu 20 dotnet SDK as base image.
It can be added to docker-compose.yml
and then used as FROM opencvsharp_base
in dockerfiles, or pushed to custom docker registry, etc.
Dockerized .NET project does not need any dependency aside from OpenCVSharp4
, native binaries will be built inside docker.
However, I'm yet to separate build environment from runtime here. To do so, one will need to copy OpenCV build output and libOpenCvSharp.so (these are currently installed directly into the container system using make install
) and install all the necessary packages (runtime should have same versions of dependencies ad build environment).
Guess that will be the next step but for now I don't mind a bit bloated container :)
Dockerfile itself:
FROM mcr.microsoft.com/dotnet/core/sdk:3.1-focal
ENV OPENCV_VERSION=4.3.0
RUN apt-get update && apt-get install -y \
apt-transport-https \
software-properties-common \
wget \
unzip \
curl \
ca-certificates
#bzip2 \
#grep sed dpkg
# Install opencv dependencies
RUN cd ~
RUN apt-get update && apt-get install -y \
build-essential \
cmake \
git \
gfortran \
libjpeg8-dev \
libpng-dev \
software-properties-common
RUN add-apt-repository "deb http://security.ubuntu.com/ubuntu xenial-security main" && apt-get update && apt-get install -y \
libjasper1 \
libtiff-dev \
libavcodec-dev \
libavformat-dev \
libswscale-dev \
libdc1394-22-dev \
libxine2-dev \
libv4l-dev
RUN cd /usr/include/linux
RUN ln -s -f ../libv4l1-videodev.h videodev.h
RUN cd ~
RUN apt-get install -y \
libgtk2.0-dev libtbb-dev qt5-default \
libatlas-base-dev \
libfaac-dev \
libmp3lame-dev \
libtheora-dev \
libvorbis-dev \
libxvidcore-dev \
libopencore-amrnb-dev \
libopencore-amrwb-dev \
libavresample-dev \
x264 \
v4l-utils
# Setup OpenCV source
RUN wget https://github.com/opencv/opencv/archive/${OPENCV_VERSION}.zip && \
unzip ${OPENCV_VERSION}.zip && \
rm ${OPENCV_VERSION}.zip && \
mv opencv-${OPENCV_VERSION} opencv
# Setup opencv-contrib Source
RUN wget https://github.com/opencv/opencv_contrib/archive/${OPENCV_VERSION}.zip && \
unzip ${OPENCV_VERSION}.zip && \
rm ${OPENCV_VERSION}.zip && \
mv opencv_contrib-${OPENCV_VERSION} opencv_contrib
# Build OpenCV
RUN cd opencv && mkdir build && cd build && \
cmake \
-D OPENCV_EXTRA_MODULES_PATH=/opencv_contrib/modules \
-D CMAKE_BUILD_TYPE=RELEASE \
-D BUILD_SHARED_LIBS=OFF \
-D ENABLE_CXX11=ON \
-D BUILD_EXAMPLES=OFF \
-D BUILD_DOCS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_JAVA=OFF \
-D BUILD_opencv_app=OFF \
-D BUILD_opencv_java=OFF \
-D BUILD_opencv_python=OFF \
-D BUILD_opencv_ts=OFF \
-D BUILD_opencv_js=OFF \
-D WITH_GSTREAMER=OFF \
-D OPENCV_ENABLE_NONFREE=ON \
.. && make -j12 && make install && ldconfig
WORKDIR /
# Download OpenCvSharp
RUN git clone https://github.com/shimat/opencvsharp.git
RUN cd opencvsharp && git fetch --all --tags --prune
# Install the Extern lib.
WORKDIR /opencvsharp/src
RUN mkdir /opencvsharp/make
RUN cd /opencvsharp/make && cmake /opencvsharp/src && make -j12 && make install
ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/share/opencv4/lib/:/usr/local/lib/:/lib/x86_64-linux-gnu/"
Example docker-compose.yml
:
version: '3.4'
services:
opencvsharp_base:
image: opencvsharp_base
build:
context: OpenCVSharpEnvironment
some_image_processing_api:
restart: unless-stopped
build:
context: .
dockerfile: src/SomeImageProcessingApi/Dockerfile
args:
configuration: Debug
depends_on:
- opencvsharp_base
Example Dockerfile
:
FROM opencvsharp_base
WORKDIR /app
COPY /src/SomeImageProcessingApi/SomeImageProcessingApi.csproj ./src/SomeImageProcessingApi/
WORKDIR /app/src/SomeImageProcessingApi
RUN dotnet restore
WORKDIR /app
COPY /src/SomeImageProcessingApi/. ./src/SomeImageProcessingApi/
WORKDIR /app/src/SomeImageProcessingApi
ARG configuration
RUN dotnet publish -c $configuration -o out
# TODO: separate runtime container
WORKDIR /app/src/SomeImageProcessingApi/out
ENTRYPOINT ["dotnet", "SomeImageProcessingApi.dll"]
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I'm using mcr.microsoft.com/dotnet/core/aspnet:3.1-bionic as runtimes image base (docker run -d -t image_id) then i execute docker exec container_name bash. The i follow the sample dockerfile step to install the depedencies and cmake. Unfortunately i face some error when build opencv
-- OpenCV modules:
-- To be built: calib3d core dnn features2d flann gapi highgui imgcodecs imgproc ml objdetect photo stitching video videoio
-- Disabled: world
-- Disabled by dependency: -
-- Unavailable: java python2 python3 ts
-- Applications: apps
-- Documentation: NO
-- Non-free algorithms: YES
--
-- GUI:
-- GTK+: YES (ver 2.24.32)
-- GThread : YES (ver 2.56.4)
-- GtkGlExt: NO
-- VTK support: NO
--
-- Media I/O:
-- ZLib: /usr/lib/x86_64-linux-gnu/libz.so (ver 1.2.11)
-- JPEG: /usr/lib/x86_64-linux-gnu/libjpeg.so (ver 80)
-- WEBP: build (ver encoder: 0x020f)
-- PNG: /usr/lib/x86_64-linux-gnu/libpng.so (ver 1.6.34)
-- TIFF: /usr/lib/x86_64-linux-gnu/libtiff.so (ver 42 / 4.0.9)
-- JPEG 2000: build (ver 2.3.1)
-- OpenEXR: build (ver 2.3.0)
-- HDR: YES
-- SUNRASTER: YES
-- PXM: YES
-- PFM: YES
--
-- Video I/O:
-- DC1394: YES (2.2.5)
-- FFMPEG: YES
-- avcodec: YES (57.107.100)
-- avformat: YES (57.83.100)
-- avutil: YES (55.78.100)
-- swscale: YES (4.8.100)
-- avresample: YES (3.7.0)
-- v4l/v4l2: YES (linux/videodev2.h)
--
-- Parallel framework: pthreads
--
-- Trace: YES (with Intel ITT)
--
-- Other third-party libraries:
-- Intel IPP: 2020.0.0 Gold [2020.0.0]
-- at: /root/opencv/build/3rdparty/ippicv/ippicv_lnx/icv
-- Intel IPP IW: sources (2020.0.0)
-- at: /root/opencv/build/3rdparty/ippicv/ippicv_lnx/iw
-- Lapack: NO
-- Eigen: NO
-- Custom HAL: NO
-- Protobuf: build (3.5.1)
--
-- OpenCL: YES (no extra features)
-- Include path: /root/opencv/3rdparty/include/opencl/1.2
-- Link libraries: Dynamic load
--
-- Python (for build): /usr/bin/python3
--
-- Install to: /usr/local
-- -----------------------------------------------------------------
--
-- Configuring incomplete, errors occurred!
See also "/root/opencv/build/CMakeFiles/CMakeOutput.log".
See also "/root/opencv/build/CMakeFiles/CMakeError.log".
i also found the opencvsharp repo have dockerfile, i build from the dockerfile then will run into the error like this:
cc1plus: out of memory allocating 1192800 bytes after a total of 34836480 bytes modules/imgproc/CMakeFiles/opencv_imgproc.dir/build.make:2056: recipe for target 'modules/imgproc/CMakeFiles/opencv_imgproc.dir/color_yuv.avx2.cpp.o' failed
When i build from the opencvsharp repo's dockerfile, i make this "shimat/ubuntu18-dotnetcore3.1-opencv4.5.0:20201030" as runtime image base.
i also found the opencvsharp repo have dockerfile, i build from the dockerfile then will run into the error like this: cc1plus: out of memory allocating 1192800 bytes after a total of 34836480 bytes modules/imgproc/CMakeFiles/opencv_imgproc.dir/build.make:2056: recipe for target 'modules/imgproc/CMakeFiles/opencv_imgproc.dir/color_yuv.avx2.cpp.o' failed
When i build from the opencvsharp repo's dockerfile, i make this "shimat/ubuntu18-dotnetcore3.1-opencv4.5.0:20201030" as runtime image base.
now have another error is like this:
c++: internal compiler error: Killed (program cc1plus)
Please submit a full bug report,
with preprocessed source if appropriate.
See <file:///usr/share/doc/gcc-7/README.Bugs> for instructions.
modules/imgproc/CMakeFiles/opencv_imgproc.dir/build.make:1984: recipe for target 'modules/imgproc/CMakeFiles/opencv_imgproc.dir/filter.avx2.cpp.o' failed
make[2]: *** [modules/imgproc/CMakeFiles/opencv_imgproc.dir/filter.avx2.cpp.o] Error 4
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:2698: recipe for target 'modules/imgproc/CMakeFiles/opencv_imgproc.dir/all' failed
make[1]: *** [modules/imgproc/CMakeFiles/opencv_imgproc.dir/all] Error 2
Makefile:162: recipe for target 'all' failed
make: *** [all] Error 2
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I'm going to write down issue, working-but-terrible solution and would like to ask more experienced people around here for better solution.
Issue
Linux binaries shipped with NuGet package are mostly unusable unless user have exactly the same environment as the one package was built in. Even if you install all the dependencies, you are most likely to end up with slightly different versions of your libraries and
OpenCVSharpExtern
won't work. So, unless you are using exactly the same Ubuntu with exactly the same versions of packages as on CI server, it won't work out of the box.Working but problematic solution
Local environment
README.md
and buildOpenCvSharpExtern
.libOpenCvSharp
(without .so) and put file into/usr/lib
or make sureld
finds it some other way.Debug with
ldd
or just read exception (it will mention what dependency is missing) if you encounter problems.Docker
Things become even more fun here. If you are hosting ASP .NET Core app inside docker linux container, the only way I found is to build
OpenCvSharpExtern
in the container similar to production and then extract compilation result along with all dependencies.mcr.microsoft.com/dotnet/core/aspnet:3.1-bionic
as runtime image base.docker exec container_name bash
to get into the container.apt-get install ...
, build OpenCV 4.3.0, build OpenCvSharp and make sure both are installed into the container system (present somewhere in/usr/local/share/opencvsharp4/
or elsewhere). There are CI scripts in this repo that should help withapt-get
part./usr/local
,/usr/lib
and/lib/x86_64-linux-gnu
which turned out to be 1.2GiB.docker-compose
automatically).Dockerfile
to copy these files from host onto runtime container after build.ENV LD_LIBRARY_PATH="${LD_LIBRARY_PATH}:/usr/local/share/opencv4/lib/:/usr/local/lib/:/lib/x86_64-linux-gnu/"
will probably be required.Questions
libOpenCvSharpExtern.so
to embed all the dependent libs into it? It looks like it was successfully done for Windows, so I wonder if it is possible on Linux.compiling OpenCV in production environment
steps on your CI server or there is better approach?I'm fairly noob in either docker and magic of C++ compilation, sorry if I'm missing something obvious. Hope this helps someone or attracts answers with better solutions :)