search

puckel / docker-airflow

Docker Apache Airflow
Apache License 2.0
3.78k stars 543 forks source link

Unable to install openJdk8 on the container #182

Open ravieze opened 6 years ago

ravieze commented 6 years ago

Version: version: '2.1' command to bring up the containers: docker-compose -f docker-compose-CeleryExecutor.yml up -d

Things are working fine till here. I am able to see the flower UI, DAG, run jobs etc. Sofar all good.

I want to use spark submit operator which requires me to have spark pakages installed -- which inturn need open jdk 8 to be present.

Now i tried all possible documents google can give me (eg: https://xmoexdev.com/wordpress/installing-openjdk-8-debian-jessie/) to install the open jdk 8 --but it was of no use. The installation always endsup in the below error. People report that they are able to install the openJDK when they use native Debian-8 (jessie); however, even though the docker container tells its Debian-8 Jessie, OPENJDk8 isnot installing.

The error i get during the installation is as follows. Any ideas how to fix this?

Setting up openjdk-8-jre-headless:amd64 (8u162-b12-1~bpo8+1) ... update-alternatives: using /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/rmid to provide /usr/bin/rmid (rmid) in auto mode update-alternatives: error: error creating symbolic link `/usr/share/man/man1/rmid.1.gz.dpkg-tmp': No such file or directory dpkg: error processing package openjdk-8-jre-headless:amd64 (--configure): subprocess installed post-installation script returned error exit status 2 dpkg: dependency problems prevent configuration of ca-certificates-java: ca-certificates-java depends on openjdk-7-jre-headless | java7-runtime-headless; however: Package openjdk-7-jre-headless is not installed. Package java7-runtime-headless is not installed. Package openjdk-8-jre-headless:amd64 which provides java7-runtime-headless is not configured yet.

dpkg: error processing package ca-certificates-java (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of openjdk-8-jre:amd64: openjdk-8-jre:amd64 depends on openjdk-8-jre-headless (= 8u162-b12-1~bpo8+1); however: Package openjdk-8-jre-headless:amd64 is not configured yet.

dpkg: error processing package openjdk-8-jre:amd64 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of openjdk-8-jdk-headless:amd64: openjdk-8-jdk-headless:amd64 depends on openjdk-8-jre-headless (= 8u162-b12-1~bpo8+1); however: Package openjdk-8-jre-headless:amd64 is not configured yet.

dpkg: error processing package openjdk-8-jdk-headless:amd64 (--configure): dependency problems - leaving unconfigured dpkg: dependency problems prevent configuration of openjdk-8-jdk:amd64: openjdk-8-jdk:amd64 depends on openjdk-8-jre (= 8u162-b12-1~bpo8+1); however: Package openjdk-8-jre:amd64 is not configured yet. openjdk-8-jdk:amd64 depends on openjdk-8-jdk-headless (= 8u162-b12-1~bpo8+1); however: Package openjdk-8-jdk-headless:amd64 is not configured yet.

dpkg: error processing package openjdk-8-jdk:amd64 (--configure): dependency problems - leaving unconfigured Setting up libgtk-3-common (3.14.5-1+deb8u1) ... Setting up libgtk-3-0:amd64 (3.14.5-1+deb8u1) ... Setting up libgtk-3-bin (3.14.5-1+deb8u1) ... Setting up adwaita-icon-theme (3.14.0-2) ... Processing triggers for systemd (230-7~bpo8+2) ... Processing triggers for libc-bin (2.19-18+deb8u10) ... Processing triggers for libgdk-pixbuf2.0-0:amd64 (2.31.1-2+deb8u7) ... Processing triggers for dbus (1.8.22-0+deb8u1) ... Processing triggers for sgml-base (1.26+nmu4) ... Errors were encountered while processing: openjdk-8-jre-headless:amd64 ca-certificates-java openjdk-8-jre:amd64 openjdk-8-jdk-headless:amd64 openjdk-8-jdk:amd64 E: Sub-process /usr/bin/dpkg returned an error code (1)

Jonathan34 commented 6 years ago

may be that will help you. This is my docker image with Java 8 installed I created my own docker image based on that one and put this inside:

FROM puckel/docker-airflow:1.9.0
COPY requirements.txt /

# install Java
USER root
RUN mkdir -p /usr/share/man/man1 && \
    (echo "deb http://http.debian.net/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list) && \
    apt-get update -y && \
    apt-get install -t jessie-backports openjdk-8-jdk -y

RUN apt-get install unzip -y && \
    apt-get autoremove -y

USER airflow
anthony-o commented 6 years ago

Thanks to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199 you just have to execute mkdir -p /usr/share/man/man1 before apt install -y openjdk-8-jdk command in order to install Open JDK 8 on Debian Stretch.

darshanmehta10 commented 5 years ago

@Jonathan34 docker image built with the above script does not start up. It doesn't log anything either. Also, when I try with the new version of the base image (i.e. 1.10.1), I get the following error:

The following packages have unmet dependencies:
 openjdk-8-jdk : Depends: openjdk-8-jre (= 8u171-b11-1~bpo8+1) but it is not going to be installed
                 Depends: openjdk-8-jdk-headless (= 8u171-b11-1~bpo8+1) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.
The command '/bin/sh -c mkdir -p /usr/share/man/man1 &&     (echo "deb http://http.debian.net/debian jessie-backports main" > /etc/apt/sources.list.d/backports.list) &&     apt-get update -y &&     apt-get install -t jessie-backports openjdk-8-jdk -y' returned a non-zero code: 100
darshanmehta10 commented 5 years ago

@Jonathan34 @anthony-o following worked for me:

FROM puckel/docker-airflow:1.10.1

# install Java
USER root
RUN mkdir -p /usr/share/man/man1 && \
    apt-get update -y && \
    apt-get install -y openjdk-8-jdk

RUN apt-get install unzip -y && \
    apt-get autoremove -y

USER airflow
Jonathan34 commented 5 years ago

@darshanmehta10 yes we updated to that couple of months ago. openjdk was not available when i posted the image above. thanks for correcting!

andrew-womeldorf commented 4 years ago

As of version 1.10.9, the base image has moved to python:3.7-slim-buster. Debian Buster appears to have removed openjdk-8 (see last message of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199)

Here's a modified version of the aforementioned Dockerfile to install openjdk-8-jdk:

FROM puckel/docker-airflow:1.10.9

# install Java
USER root
RUN echo "deb http://security.debian.org/debian-security stretch/updates main" >> /etc/apt/sources.list                                                   
RUN mkdir -p /usr/share/man/man1 && \
    apt-get update -y && \
    apt-get install -y openjdk-8-jdk

RUN apt-get install unzip -y && \
    apt-get autoremove -y

USER airflow

There may be a better repository to add, but this one works.

chencivalue commented 4 years ago

As of version 1.10.9, the base image has moved to python:3.7-slim-buster. Debian Buster appears to have removed openjdk-8 (see last message of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199)

Here's a modified version of the aforementioned Dockerfile to install openjdk-8-jdk:

FROM puckel/docker-airflow:1.10.9

# install Java
USER root
RUN echo "deb http://security.debian.org/debian-security stretch/updates main" >> /etc/apt/sources.list                                                   
RUN mkdir -p /usr/share/man/man1 && \
    apt-get update -y && \
    apt-get install -y openjdk-8-jdk

RUN apt-get install unzip -y && \
    apt-get autoremove -y

USER airflow

There may be a better repository to add, but this one works.

It works! thanks!

kantharajucn commented 4 years ago

As of version 1.10.9, the base image has moved to python:3.7-slim-buster. Debian Buster appears to have removed openjdk-8 (see last message of https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199)

Here's a modified version of the aforementioned Dockerfile to install openjdk-8-jdk:

FROM puckel/docker-airflow:1.10.9

# install Java
USER root
RUN echo "deb http://security.debian.org/debian-security stretch/updates main" >> /etc/apt/sources.list                                                   
RUN mkdir -p /usr/share/man/man1 && \
    apt-get update -y && \
    apt-get install -y openjdk-8-jdk

RUN apt-get install unzip -y && \
    apt-get autoremove -y

USER airflow

There may be a better repository to add, but this one works.

This works. Thanks:)

rbreaves commented 2 years ago

Thanks to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199 you just have to execute mkdir -p /usr/share/man/man1 before apt install -y openjdk-8-jdk command in order to install Open JDK 8 on Debian Stretch.

Just got nailed by this.. and yep creating that folder solves it.

sephethus commented 1 year ago

Thanks to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=863199 you just have to execute mkdir -p /usr/share/man/man1 before apt install -y openjdk-8-jdk command in order to install Open JDK 8 on Debian Stretch.

I've seen this answer given twice, not the answer sorry.