search

eclipse-che / che

Kubernetes based Cloud Development Environments for Enterprise Teams
http://eclipse.org/che
Eclipse Public License 2.0
6.99k stars 1.19k forks source link

cannot create WS from a docker file #1006

Closed itaimalek closed 8 years ago

itaimalek commented 8 years ago

Hi, I've installed Docker and che natively on a remote machine (che as a server), I'm able to create WS from the given stack library but cannot do so from a docker file using custom stack interface, I've took the Dockerfile from the dockerfile repository described in che docs : https://github.com/codenvy/dockerfiles/ the centos Java8 Docker file is here: https://github.com/codenvy/dockerfiles/blob/master/centos_jdk8/Dockerfile

and this is it's content:

FROM centos
EXPOSE 4403 8080 8000 22
RUN yum update -y && \
    yum -y install sudo openssh-server procps wget unzip mc git curl subversion nmap && \
    mkdir /var/run/sshd && \
    sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd && \
    echo "%wheel ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
    useradd -u 1000 -G users,wheel -d /home/user --shell /bin/bash -m user -p secret && \
    sed -i 's/requiretty/!requiretty/g' /etc/sudoers

USER user

LABEL che:server:8080:ref=tomcat8 che:server:8080:protocol=http che:server:8000:ref=tomcat8-debug che:server:8000:protocol=http

ENV MAVEN_VERSION=3.2.2 \
    JAVA_VERSION=8u45 \
    JAVA_VERSION_PREFIX=1.8.0_45 \
    TOMCAT_HOME=/home/user/tomcat8

ENV JAVA_HOME=/opt/jdk$JAVA_VERSION_PREFIX \
    M2_HOME=/opt/apache-maven-$MAVEN_VERSION

ENV PATH=$JAVA_HOME/bin:$M2_HOME/bin:$PATH

RUN mkdir /home/user/tomcat8 && sudo mkdir /opt/apache-maven-$MAVEN_VERSION && \
  wget \
  --no-cookies \
  --no-check-certificate \
  --header "Cookie: oraclelicense=accept-securebackup-cookie" \
  -qO- \
  "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-b14/jdk-$JAVA_VERSION-linux-x64.tar.gz" | sudo tar -zx -C /opt/ && \
  sudo wget -qO- "https://archive.apache.org/dist/maven/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz" | sudo tar -zx --strip-components=1 -C /opt/apache-maven-$MAVEN_VERSION/

ENV TERM xterm

RUN wget -qO- "http://archive.apache.org/dist/tomcat/tomcat-8/v8.0.24/bin/apache-tomcat-8.0.24.tar.gz" | tar -zx --strip-components=1 -C /home/user/tomcat8 && \
    rm -rf /home/user/tomcat8/webapps/*
USER root
RUN  echo -e "JAVA_HOME=/opt/jdk$JAVA_VERSION_PREFIX\nM2_HOME=/home/user/apache-maven-$MAVEN_VERSION\nTOMCAT_HOME=/home/user/tomcat8\nPATH=$JAVA_HOME/bin:$M2_HOME/bin:$PATH" >> /etc/environment
USER user
ENV LANG C.UTF-8
CMD sudo /usr/sbin/sshd -D && \
    tail -f /dev/null

and pasted it here: image

When trying to create the container I get an error from the server:

[DOCKER] [ERROR] The command '/bin/sh -c mkdir /home/user/tomcat8 && sudo mkdir /opt/apache-maven-$MAVEN_VERSION && wget --no-cookies --no-check-certificate --header "Cookie: oraclelicense=accept-securebackup-cookie" -qO- "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-b14/jdk-$JAVA_VERSION-linux-x64.tar.gz" | sudo tar -zx -C /opt/ && sudo wget -qO- "https://archive.apache.org/dist/maven/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz" | sudo tar -zx --strip-components=1 -C /opt/apache-maven-$MAVEN_VERSION/' returned a non-zero code: 2
[ERROR] Docker image build failed

What went wrong? What is the difference when running it from the stack library or running it from Dockerfile?

Im new both to Docker and Che so excuse my noobness....

Also-

  1. on repeated attempts it seems che is trying to build the WS from cache - how can I clean this cache?
  2. when trying to create with a simple Docker file like: FROM ubuntu I get 
Error when starting agent
Unable to start workspace agent. Error when trying to start the workspace agent: Error response from docker API, status: 500, message: Cannot kill container addfb8e8a256e4388a2536a5a8599c2530cd388ba3352b5e9adea79f2957db46: Container addfb8e8a256e4388a2536a5a8599c2530cd388ba3352b5e9adea79f2957db46 is not running
ghost commented 8 years ago

Hi

I have just built this image both locally and at DockerHub (automated build). Looks like the instruction that failed has been caused by network issues.

You do not need to build our base images, since using FROM codenvy/baseImage is the best way to go.

As to why FROM ubuntu fails, this dockerfile does not have any CMD. So, after docker has executed all instructions, the container enters an exited state. This is why we have tailf /dev/null in all CMDs.

Besides, FROM ubuntu won't work since it does not have java which a workspace agent requires (Tomcat8)

itaimalek commented 8 years ago

So, basically all images managed by Che must be provisioned with java and Tomcat8 ? which java version is required? would it work with java 7_75?

Also, my environment needs to be able to run Tomcat 6 as this is a prerequisite for our dev product. would I need to figure out a way of running both Tomcat 8 and 6?

TylerJewell commented 8 years ago

Hiya. The images managed by Che - we call those custom stacks. So you can see what we put into those stacks and their requirements here.

The Tomcat that is running inside of the images hosts our agents on a certain port. It is ok to run other services, such as different Tomcat versions within the container at the same time.

ghost commented 8 years ago

Workspace agent (packaged as Tomcat 8) requires Java 1.8. As to requirements to custom stacks, you may want to take a look at docs - https://eclipse-che.readme.io/docs/stacks#custom-workspace-recipes

itaimalek commented 8 years ago

ok, now I just need to figure out a way to use both Java 7 and Java 8 on the same machine... Thanks guys :)

ghost commented 8 years ago

It is possible. You need to download both Javas, write JAVA_HOME for Java 1.8 to ~/.bashrc. After a workspace has booted, rewrite JAVA_HOME to Java 1.7.

itaimalek commented 8 years ago

sounds good, I'll try. thanks.

BTW i've corrupted my env. , installed everything again and now...can't create WS... I've opened a new thread here: https://github.com/eclipse/che/issues/1017

itaimalek commented 8 years ago

ok, now that I can create environments ... :)

I added this to the CentOS/Java8 Dockerfile :

RUN wget \
    --no-check-certificate \
    --no-cookies \
    --header "Cookie: oraclelicense=accept-securebackup-cookie" \
    -qO- \
    "http://download.oracle.com/otn-pub/java/jdk/7u75-b13/jdk-7u75-linux-x64.tar.gz" | sudo tar -zx -C /opt/ && \

just above the CMD line, but the container fails to build... any idea why?

ghost commented 8 years ago

No idea until I see logs. Can you share docker build logs please?

itaimalek commented 8 years ago

Where can I find them? On Apr 13, 2016 9:56 AM, "Eugene Ivantsov" notifications@github.com wrote:

No idea until I see logs. Can you share docker build logs please?

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/eclipse/che/issues/1006#issuecomment-209261649

ghost commented 8 years ago

@itaimalek when starting a workspace from a custom recipe you should see the output.

itaimalek commented 8 years ago

problem was : && \ at the end while this is the end of file and && \ implies continuation. thanks for your help :)

itaimalek commented 8 years ago

What is the build context path for custom stacks? where should I put files for COPY in dockerfile?

ddementieva commented 8 years ago

There is no build context path when machine API communicates with Docker API.

As a workaround, you can add a remotely hosted file to a Docker image with the same ADD instruction. Use file URL instead of its path.

itaimalek commented 8 years ago

ok, I've put my files in a remote storage and it seems I can pull the with ADD, right now I;m getting this right at the end of the build : 

Error when starting agent
Unable to start workspace agent. Error when trying to start the workspace agent: Error response from docker API, status: 500, message: Cannot start container 9ea09253332bc524aa671dea4f11a66cefd63c289610ee3e19ac1b4db010b62f: [9] System error: not a directory

my Dockerfile is this:

FROM centos
EXPOSE 4403 8080 8000 22
RUN yum update -y && \
    yum -y install sudo openssh-server procps wget unzip mc git curl subversion nmap && \
    mkdir /var/run/sshd && \
    sed 's@session\s*required\s*pam_loginuid.so@session optional pam_loginuid.so@g' -i /etc/pam.d/sshd && \
    echo "%wheel ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers && \
    useradd -u 1000 -G users,wheel -d /home/user --shell /bin/bash -m user -p secret && \
    sed -i 's/requiretty/!requiretty/g' /etc/sudoers

USER user

LABEL che:server:8080:ref=tomcat8 che:server:8080:protocol=http che:server:8000:ref=tomcat8-debug che:server:8000:protocol=http

ENV MAVEN_VERSION=3.2.2 \
    JAVA_VERSION=8u45 \
    JAVA_VERSION_PREFIX=1.8.0_45 \
    TOMCAT_HOME=/home/user/tomcat8

ENV JAVA_HOME=/opt/jdk$JAVA_VERSION_PREFIX \
    M2_HOME=/opt/apache-maven-$MAVEN_VERSION

ENV PATH=$JAVA_HOME/bin:$M2_HOME/bin:$PATH

RUN mkdir /home/user/tomcat8 && sudo mkdir /opt/apache-maven-$MAVEN_VERSION && \
  wget \
  --no-cookies \
  --no-check-certificate \
  --header "Cookie: oraclelicense=accept-securebackup-cookie" \
  -qO- \
  "http://download.oracle.com/otn-pub/java/jdk/$JAVA_VERSION-b14/jdk-$JAVA_VERSION-linux-x64.tar.gz" | sudo tar -zx -C /opt/ && \
  sudo wget -qO- "https://archive.apache.org/dist/maven/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz" | sudo tar -zx --strip-components=1 -C /opt/apache-maven-$MAVEN_VERSION/

ENV TERM xterm

RUN wget -qO- "http://archive.apache.org/dist/tomcat/tomcat-8/v8.0.24/bin/apache-tomcat-8.0.24.tar.gz" | tar -zx --strip-components=1 -C /home/user/tomcat8 && \
    rm -rf /home/user/tomcat8/webapps/*
USER root
RUN  echo -e "JAVA_HOME=/opt/jdk$JAVA_VERSION_PREFIX\nM2_HOME=/home/user/apache-maven-$MAVEN_VERSION\nTOMCAT_HOME=/home/user/tomcat8\nPATH=$JAVA_HOME/bin:$M2_HOME/bin:$PATH" >> /etc/environment
USER user
ENV LANG C.UTF-8

WORKDIR /tmp

RUN wget \
    --no-check-certificate \
    --no-cookies \
    --header "Cookie: oraclelicense=accept-securebackup-cookie" \
    -qO- \
    "http://download.oracle.com/otn-pub/java/jdk/7u75-b13/jdk-7u75-linux-x64.tar.gz" | sudo tar -zx -C /opt/ 

ADD https://console.cloud.google.com/m/cloudstorage/b/gciassets/o/liferay-portal-tomcat-6.1.1-ce-ga2.zip  /projects

RUN sudo wget -qO- "http://www.us.apache.org/dist/ant/binaries/apache-ant-1.9.6-bin.tar.gz" |sudo tar -zx -C /opt/

ENV ANT_HOME = /opt/apache-ant-1.9.6\
    PATH=$ANT_HOME/bin:$PATH

USER root
RUN  ln -s /opt/apache-ant-1.9.6/bin/ant /usr/bin/ant

USER user
WORKDIR /tmp

RUN sudo wget -qO- "http://nodejs.org/dist/v0.12.0/node-v0.12.0-linux-x64.tar.gz" | sudo tar -zx -C /opt/
ENV NODE_HOME=/opt/node-v0.12.0-linux-x64 \
    PATH=$NODE_HOME/bin:$PATH  
USER root   
RUN  ln -s /opt/node-v0.12.0-linux-x64/bin/node /usr/bin/node &&\
     ln -s /opt/node-v0.12.0-linux-x64/lib/node /usr/lib/node &&\
     ln -s /opt/node-v0.12.0-linux-x64/bin/npm /usr/bin/npm &&\
     ln -s /opt/node-v0.12.0-linux-x64/bin/node-waf /usr/bin/node-waf

USER user
CMD sudo /usr/sbin/sshd -D && \
    tail -f /dev/null

output from che console UI:

[DOCKER] ---> Running in 16289db549c8
[DOCKER] ---> 0b032ac6be62
[DOCKER] Removing intermediate container 16289db549c8
[DOCKER] Step 27 : USER user
[DOCKER] ---> Running in b29f70c4d145
[DOCKER] ---> 6658d92dd20e
[DOCKER] Removing intermediate container b29f70c4d145
[DOCKER] Step 28 : CMD sudo /usr/sbin/sshd -D && tail -f /dev/null
[DOCKER] ---> Running in b94e99f3911e
[DOCKER] ---> 2584ecd32780
[DOCKER] Removing intermediate container b94e99f3911e
[DOCKER] Successfully built 2584ecd32780
[ERROR] Error response from docker API, status: 500, message: Cannot start container 9ea09253332bc524aa671dea4f11a66cefd63c289610ee3e19ac1b4db010b62f: [9] System error: not a directory

What went wrong?

ghost commented 8 years ago

Can you build an image from this Dockerfile locally? Without using Che?

itaimalek commented 8 years ago

ok, problem was the ADD statement trying to add a file to the /project dir, while the Dockerfile is being executed this dir is not yet exist. in what stage does che add this dir?

ghost commented 8 years ago

projects dir gets created when a workspace starts. This is where workspace projects get mounted.

itaimalek commented 8 years ago

so, no option to copy files to it prior to IDE start, right?

ghost commented 8 years ago

Why? /home/user is a good place to add files. Just bear in mind that files ADDed to an image are added with root permissions and you need to run chown on them.

itaimalek commented 8 years ago

Right, but no access to /projects from the Dockerfile, correct? On Apr 14, 2016 5:50 PM, "Eugene Ivantsov" notifications@github.com wrote:

Why? /home/user is a good place to add files. Just bear in mind that files ADDed to an image are added with root permissions and you need to run chown on them.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/eclipse/che/issues/1006#issuecomment-209983638

ghost commented 8 years ago

In your CMD instruction you may access /projects dir. For instance, copy smth from /home/user to /projects