Closed keremceliker closed 4 years ago
Hey,
I cannot remember exactly what the issue was but I can share you my current DC & BC. I am running it with the default ServiceAccount which has the permission to create privileged pods
apiVersion: build.openshift.io/v1
metadata:
labels:
app: ${APPLICATION_NAME}
name: ${APPLICATION_NAME}
spec:
nodeSelector:
output:
to:
kind: ImageStreamTag
name: ${APPLICATION_NAME}:latest
runPolicy: Serial
source:
binary: {}
type: Binary
strategy:
dockerStrategy: {}
apiVersion: apps.openshift.io/v1
metadata:
name: ${APPLICATION_NAME}
labels:
app: ${APPLICATION_NAME}
spec:
strategy:
type: Rolling
triggers:
- type: ImageChange
imageChangeParams:
automatic: true
containerNames:
- ${APPLICATION_NAME}
from:
kind: ImageStream
name: ${APPLICATION_NAME}:latest
- type: ConfigChange
replicas: 1
selector:
deploymentConfig: ${APPLICATION_NAME}
template:
metadata:
name: ${APPLICATION_NAME}
labels:
deploymentConfig: ${APPLICATION_NAME}
app: ${APPLICATION_NAME}
spec:
terminationGracePeriodSeconds: 60
serviceAccountName: default
securityContext:
runAsUser: 0
containers:
- name: ${APPLICATION_NAME}
image: ${APPLICATION_NAME}:latest
imagePullPolicy: Always
ports:
- name: http
containerPort: 8080
protocol: TCP
env:
- name: "NIFI_JAVA_XMS"
value: ${NIFI_JAVA_XMS}
- name: "NIFI_JAVA_XMX"
value: ${NIFI_JAVA_XMX}
- name: "NIFI_TIMEZONE"
value: ${NIFI_TIMEZONE}
- name: "ENVIRONMENT"
value: ${TARGET_ENVIRONMENT}
- name: "HOME"
value: /tmp
resources:
limits:
cpu: ${CPU_LIMIT}
memory: ${MEMORY_LIMIT}
requests:
cpu: ${CPU_REQUEST}
memory: ${MEMORY_REQUEST}
volumeMounts:
volumes:
Hey Mate, i tried the build it on MiniShift and i got the follows error in POD now while pod creation processing.
Any idea on it to fix ?
container_linux.go:235: starting container process caused "exec: \"../scripts/start.sh\": permission denied"
--
Kerem Çeliker
Hey Mate, i tried the build it on MiniShift and i got the follows error in POD now while pod creation processing.
Any idea on it to fix ?
container_linux.go:235: starting container process caused "exec: \"../scripts/start.sh\": permission denied" --
Hey,
im not at home atm. That error states you need to chmod that file in the Dockerfile for ease you cam use „chmod 777 /scriptdir/start.sh“
Cheers
Hey Mate, I just added as you mentioned chmod 777.. in Dockerfile but im still getting error while im build the code.
Error code the follows below;
chmod: cannot access '/scripts/start.sh': No such file or directory
Removing intermediate container 0fd3ad71a841
The command '/bin/sh -c chmod 777 /scripts/start.sh' returned a non-zero code: 1
Could you edit for me mate, according to what I share below ?
Here is the Currently Dockerfile:
ADD sh/ ${NIFI_BASE_DIR}/scripts/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_HOME}
ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000
WORKDIR ${NIFI_HOME}
USER 1001
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
RUN chmod 777 start.sh
ENTRYPOINT ["../scripts/start.sh"]
Kerem Çeliker
You did not specify the correct path for the start.sh script Its more like:
ARG NIFI_VERSION=1.9.2
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
USER root
ADD sh/ ${NIFI_BASE_DIR}/scripts/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps python3
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_HOME}
ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
RUN chmod 777 ../scripts/start.sh
RUN chmod 777 /opt/nifi/scripts/toolkit.sh
USER 1001
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
ENTRYPOINT ["../scripts/start.sh"]
Thanks Mate !, u re really rock. it works well according to my another try but there seems a little issue in POD while the creating on it that we got the follows. Any idea what should i do ?
Here is the Issue (after run your code,mate)
replacing target file /opt/nifi/nifi-current/conf/nifi.properties
--
|/opt/nifi/scripts/toolkit.sh: 18: /opt/nifi/scripts/toolkit.sh: cannot create //.nifi-cli.nifi.properties: Permission denied
The problem has been Fixed... The reason for the problem is the permission issue caused by the Dockerfile that came in the catalog. Generally a bug Apache NiFi-sourced on OpenShift when u want to run on it with Docker Image...
Kerem Çeliker
Hey @keremceliker
could you share your fix, please?
I am stuck with the same problem...
Thanks, Rosko
Hey @keremceliker
could you share your fix, please?
I am stuck with the same problem...
Thanks, Rosko
Did you add the ServiceAccount with priviliged SCC ? Also did you put RunAsUser in the Template?
serviceAccountName: default
securityContext:
runAsUser: 0
Hi @wilddeuces ,
thanks for your response! Unfortunately, this is not an option, because it's not permitted on the OpenShift cluster where I am running.
Hi @wilddeuces ,
thanks for your response! Unfortunately, this is not an option, because it's not permitted on the OpenShift cluster where I am running.
Try that solution:
Run with default ServiceAccount (without RunAsUser 0)
Dockerfile:
FROM openjdk:8-jre
#ARG NIFI_VERSION=1.9.2
ARG NIFI_VERSION=1.12.1
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
USER root
RUN chmod -R 777 /opt/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps python3 bash nano
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_HOME}
USER 1001
ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
ADD sh/ ${NIFI_BASE_DIR}/scripts/
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
RUN chmod 777 ../scripts/start.sh
RUN chmod 777 /opt/nifi/scripts/toolkit.sh
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
ENTRYPOINT ["../scripts/start.sh"]
Hey @wilddeuces,
Thank you for your persistence, but that also did not work.
Step 23/25 : RUN chmod 777 ../scripts/start.sh ---> Running in c3847fdd7ddc [91mchmod: changing permissions of '../scripts/start.sh': Operation not permitted [0m Removing intermediate container c3847fdd7ddc The command '/bin/sh -c chmod 777 ../scripts/start.sh' returned a non-zero code: 1
What OpenShift version are you using? This should run fine on v4.5 Try adding the scripts while building with root:
FROM openjdk:8-jre
#ARG NIFI_VERSION=1.9.2
ARG NIFI_VERSION=1.12.1
ARG BASE_URL=https://archive.apache.org/dist
ARG MIRROR_BASE_URL=${MIRROR_BASE_URL:-${BASE_URL}}
ARG NIFI_BINARY_PATH=${NIFI_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-${NIFI_VERSION}-bin.zip}
ARG NIFI_TOOLKIT_BINARY_PATH=${NIFI_TOOLKIT_BINARY_PATH:-/nifi/${NIFI_VERSION}/nifi-toolkit-${NIFI_VERSION}-bin.zip}
ENV NIFI_BASE_DIR=/opt/nifi
ENV NIFI_HOME ${NIFI_BASE_DIR}/nifi-current
ENV NIFI_TOOLKIT_HOME ${NIFI_BASE_DIR}/nifi-toolkit-current
ENV NIFI_PID_DIR=${NIFI_HOME}/run
ENV NIFI_LOG_DIR=${NIFI_HOME}/logs
USER root
ADD sh/ ${NIFI_BASE_DIR}/scripts/
RUN chmod -R 777 /opt/
# Setup NiFi user and create necessary directories
RUN mkdir -p ${NIFI_BASE_DIR} \
&& apt-get update \
&& apt-get install -y jq xmlstarlet procps python3 bash nano
# Download, validate, and expand Apache NiFi Toolkit binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_TOOLKIT_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} ${NIFI_TOOLKIT_HOME} \
&& ln -s ${NIFI_TOOLKIT_HOME} ${NIFI_BASE_DIR}/nifi-toolkit-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_TOOLKIT_HOME}
# Download, validate, and expand Apache NiFi binary.
RUN curl -fSL ${MIRROR_BASE_URL}/${NIFI_BINARY_PATH} -o ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& echo "$(curl ${BASE_URL}/${NIFI_BINARY_PATH}.sha256) *${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip" | sha256sum -c - \
&& unzip ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip -d ${NIFI_BASE_DIR} \
&& rm ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION}-bin.zip \
&& mv ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} ${NIFI_HOME} \
&& mkdir -p ${NIFI_HOME}/conf \
&& mkdir -p ${NIFI_HOME}/database_repository \
&& mkdir -p ${NIFI_HOME}/flowfile_repository \
&& mkdir -p ${NIFI_HOME}/content_repository \
&& mkdir -p ${NIFI_HOME}/provenance_repository \
&& mkdir -p ${NIFI_HOME}/state \
&& mkdir -p ${NIFI_LOG_DIR} \
&& ln -s ${NIFI_HOME} ${NIFI_BASE_DIR}/nifi-${NIFI_VERSION} \
&& chmod -R g+rwX ${NIFI_HOME}
ADD bootstrap.conf ${NIFI_HOME}/conf/bootstrap.conf
USER 1001
# Clear nifi-env.sh in favour of configuring all environment variables in the Dockerfile
RUN echo "#!/bin/sh\n" > ${NIFI_HOME}/bin/nifi-env.sh
# Web HTTP(s) & Socket Site-to-Site Ports
EXPOSE 8080 8443 10000 8000
WORKDIR ${NIFI_HOME}
# Apply configuration and start NiFi
#
# We need to use the exec form to avoid running our command in a subshell and omitting signals,
# thus being unable to shut down gracefully:
# https://docs.docker.com/engine/reference/builder/#entrypoint
#
# Also we need to use relative path, because the exec form does not invoke a command shell,
# thus normal shell processing does not happen:
# https://docs.docker.com/engine/reference/builder/#exec-form-entrypoint-example
ENTRYPOINT ["../scripts/start.sh"]
Could you share what you did as Fix in Deployment Config on it ?
Had to fix the given DeploymentConfig
Originally posted by @wilddeuces in https://github.com/rromannissen/nifi-openshift/issues/2#issuecomment-527427193