search

sclorg / httpd-container

Apache HTTP container images based on Red Hat Software Collections and intended for OpenShift and general usage. Users can choose between Red Hat Enterprise Linux, Fedora, and CentOS based images.
http://softwarecollections.org
Apache License 2.0
49 stars 122 forks source link

BZ1993439 - Changing way how httpd-pre-init is sourced to allow usage of ConfigMaps as source for httpd-pre-init #139

Closed sreber84 closed 2 years ago

sreber84 commented 2 years ago

This change does a small modification how files in httpd-pre-init are identified and process to allow processing scripts provided via ConfigMap in OpenShift Container Platform.

Previous functionality should not be impacted as things still work as expected, but can now be hierarchically build if needed. The below shows a regular Source-to-Image build in OpenShift Container Platform 4 to show-case that the change does not impact existing functionality.

$ oc get bc
NAME    TYPE     FROM   LATEST
httpd   Source   Git    6
[sreber@housi openshift4]$ oc get bc httpd -o yaml
apiVersion: build.openshift.io/v1
kind: BuildConfig
metadata:
  annotations:
    app.openshift.io/vcs-ref: ""
    app.openshift.io/vcs-uri: https://github.com/sreber84/httpd-ex.git
    openshift.io/generated-by: OpenShiftWebConsole
  creationTimestamp: "2022-03-29T05:07:10Z"
  generation: 7
  labels:
    app: httpd
    app.kubernetes.io/component: httpd
    app.kubernetes.io/instance: httpd
    app.kubernetes.io/name: httpd
    app.kubernetes.io/part-of: httpd
    app.openshift.io/runtime: httpd
    app.openshift.io/runtime-version: rhbz1993439
  name: httpd
  namespace: project-202
  resourceVersion: "62920439"
  uid: 5f32265b-e6b9-45e0-8e7a-f3491da4cd00
spec:
  failedBuildsHistoryLimit: 5
  nodeSelector: null
  output:
    to:
      kind: ImageStreamTag
      name: httpd:latest
  postCommit: {}
  resources: {}
  runPolicy: Serial
  source:
    contextDir: /
    git:
      uri: https://github.com/sreber84/httpd-ex.git
    type: Git
  strategy:
    sourceStrategy:
      from:
        kind: ImageStreamTag
        name: httpd:rhbz1993439
        namespace: openshift
    type: Source
  successfulBuildsHistoryLimit: 5
  triggers:
  - generic:
      secretReference:
        name: httpd-generic-webhook-secret
    type: Generic
  - github:
      secretReference:
        name: httpd-github-webhook-secret
    type: GitHub
  - imageChange: {}
    type: ImageChange
  - type: ConfigChange
status:
  imageChangeTriggers:
  - from:
      name: httpd:rhbz1993439
      namespace: openshift
    lastTriggerTime: "2022-03-29T06:48:29Z"
    lastTriggeredImageID: quay.io/rhn_support_sreber/httpd@sha256:73b823d656a4452380e281b6b0ae688a2d021a20580f7f6ad2c4a82adc28c8db
  lastVersion: 6
$ oc get build httpd-6 -o yaml
apiVersion: build.openshift.io/v1
kind: Build
metadata:
  annotations:
    openshift.io/build-config.name: httpd
    openshift.io/build.number: "6"
    openshift.io/build.pod-name: httpd-6-build
  creationTimestamp: "2022-03-29T06:48:29Z"
  generation: 2
  labels:
    app: httpd
    app.kubernetes.io/component: httpd
    app.kubernetes.io/instance: httpd
    app.kubernetes.io/name: httpd
    app.kubernetes.io/part-of: httpd
    app.openshift.io/runtime: httpd
    app.openshift.io/runtime-version: rhbz1993439
    buildconfig: httpd
    openshift.io/build-config.name: httpd
    openshift.io/build.start-policy: Serial
  name: httpd-6
  namespace: project-202
  ownerReferences:
  - apiVersion: build.openshift.io/v1
    controller: true
    kind: BuildConfig
    name: httpd
    uid: 5f32265b-e6b9-45e0-8e7a-f3491da4cd00
  resourceVersion: "62921379"
  uid: eb3ffabf-b3fd-4dc2-9d47-07aa1f9f4e74
spec:
  nodeSelector: null
  output:
    pushSecret:
      name: builder-dockercfg-krq9h
    to:
      kind: ImageStreamTag
      name: httpd:latest
  postCommit: {}
  resources: {}
  revision:
    git:
      author:
        email: sreber84@users.noreply.github.com
        name: Simon Reber
      commit: 98174f62e8739d99bd42d6e81932146e0caab0be
      committer:
        email: noreply@github.com
        name: GitHub
      message: Script 2 added
    type: Git
  serviceAccount: builder
  source:
    contextDir: /
    git:
      uri: https://github.com/sreber84/httpd-ex.git
    type: Git
  strategy:
    sourceStrategy:
      from:
        kind: DockerImage
        name: quay.io/rhn_support_sreber/httpd@sha256:73b823d656a4452380e281b6b0ae688a2d021a20580f7f6ad2c4a82adc28c8db
    type: Source
  triggeredBy:
  - imageChangeBuild:
      fromRef:
        kind: ImageStreamTag
        name: httpd:rhbz1993439
        namespace: openshift
      imageID: quay.io/rhn_support_sreber/httpd@sha256:73b823d656a4452380e281b6b0ae688a2d021a20580f7f6ad2c4a82adc28c8db
    message: Image change
status:
  completionTimestamp: "2022-03-29T06:48:53Z"
  conditions:
  - lastTransitionTime: "2022-03-29T06:48:29Z"
    lastUpdateTime: "2022-03-29T06:48:29Z"
    status: "False"
    type: New
  - lastTransitionTime: "2022-03-29T06:48:31Z"
    lastUpdateTime: "2022-03-29T06:48:31Z"
    status: "False"
    type: Pending
  - lastTransitionTime: "2022-03-29T06:48:53Z"
    lastUpdateTime: "2022-03-29T06:48:53Z"
    status: "False"
    type: Running
  - lastTransitionTime: "2022-03-29T06:48:53Z"
    lastUpdateTime: "2022-03-29T06:48:53Z"
    status: "True"
    type: Complete
  config:
    kind: BuildConfig
    name: httpd
    namespace: project-202
  duration: 24000000000
  output:
    to:
      imageDigest: sha256:870dcd1c3bae29a13c8a51b4c3ff9fdddced0cf124f201e24f735661d5480e74
  outputDockerImageReference: image-registry.openshift-image-registry.svc:5000/project-202/httpd:latest
  phase: Complete
  stages:
  - durationMilliseconds: 537
    name: FetchInputs
    startTime: "2022-03-29T06:48:31Z"
    steps:
    - durationMilliseconds: 537
      name: FetchGitSource
      startTime: "2022-03-29T06:48:31Z"
  - durationMilliseconds: 11528
    name: PullImages
    startTime: "2022-03-29T06:48:33Z"
    steps:
    - durationMilliseconds: 11528
      name: PullBaseImage
      startTime: "2022-03-29T06:48:33Z"
  - durationMilliseconds: 4958
    name: Build
    startTime: "2022-03-29T06:48:45Z"
    steps:
    - durationMilliseconds: 4958
      name: DockerBuild
      startTime: "2022-03-29T06:48:45Z"
  - durationMilliseconds: 2365
    name: PushImage
    startTime: "2022-03-29T06:48:50Z"
    steps:
    - durationMilliseconds: 2365
      name: PushImage
      startTime: "2022-03-29T06:48:50Z"
  startTimestamp: "2022-03-29T06:48:29Z"
$ oc get deployment httpd -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    alpha.image.policy.openshift.io/resolve-names: '*'
    app.openshift.io/vcs-ref: ""
    app.openshift.io/vcs-uri: https://github.com/sreber84/httpd-ex.git
    deployment.kubernetes.io/revision: "6"
    image.openshift.io/triggers: '[{"from":{"kind":"ImageStreamTag","name":"httpd:latest","namespace":"project-202"},"fieldPath":"spec.template.spec.containers[?(@.name==\"httpd\")].image","pause":"false"}]'
    openshift.io/generated-by: OpenShiftWebConsole
  creationTimestamp: "2022-03-29T05:07:10Z"
  generation: 14
  labels:
    app: httpd
    app.kubernetes.io/component: httpd
    app.kubernetes.io/instance: httpd
    app.kubernetes.io/name: httpd
    app.kubernetes.io/part-of: httpd
    app.openshift.io/runtime: httpd
    app.openshift.io/runtime-version: rhbz1993439
  name: httpd
  namespace: project-202
  resourceVersion: "62921508"
  uid: 49d8e0f0-88dd-4a54-a391-40f1b9a3abb2
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: httpd
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: httpd
        deploymentconfig: httpd
    spec:
      containers:
      - image: image-registry.openshift-image-registry.svc:5000/project-202/httpd@sha256:870dcd1c3bae29a13c8a51b4c3ff9fdddced0cf124f201e24f735661d5480e74
        imagePullPolicy: Always
        name: httpd
        ports:
        - containerPort: 8080
          protocol: TCP
        - containerPort: 8443
          protocol: TCP
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2022-03-29T06:46:56Z"
    lastUpdateTime: "2022-03-29T06:46:56Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  - lastTransitionTime: "2022-03-29T05:07:10Z"
    lastUpdateTime: "2022-03-29T06:48:56Z"
    message: ReplicaSet "httpd-84d7895c77" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  observedGeneration: 14
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

The resulting pod is showing expected behavior and running both scripts created in the git repository

$ oc logs httpd-84d7895c77-6s6f5
=> sourcing /opt/app-root/src/httpd-pre-init/script.sh ...
Test 1 script start ...
Test 1 script end ...
=> sourcing /opt/app-root/src/httpd-pre-init/script2.sh ...
Test 2 script start ...
Test 2 script end ...
=> sourcing [10-set-mpm.sh](http://10-set-mpm.sh/) ...
=> sourcing [20-copy-config.sh](http://20-copy-config.sh/) ...
=> sourcing [40-ssl-certs.sh](http://40-ssl-certs.sh/) ...
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using [10.129.2.35](http://10.129.2.35/). Set the 'ServerName' directive globally to suppress this message
[Tue Mar 29 06:48:55.943438 2022] [ssl:warn] [pid 1:tid 139883805257152] AH01909: [10.129.2.35](http://10.129.2.35/):8443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 29 06:48:55.943606 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/) configured.
[Tue Mar 29 06:48:55.943614 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: APR compiled version="1.6.3"; loaded version="1.6.3"
[Tue Mar 29 06:48:55.943617 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: PCRE compiled version="8.42 "; loaded version="8.42 2018-03-20"
[Tue Mar 29 06:48:55.943622 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: LUA compiled version="Lua 5.3"
[Tue Mar 29 06:48:55.943624 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: YAJL compiled version="2.1.0"
[Tue Mar 29 06:48:55.943626 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: LIBXML compiled version="2.9.7"
[Tue Mar 29 06:48:55.943628 2022] [:notice] [pid 1:tid 139883805257152] ModSecurity: Status engine is currently disabled, enable it by set SecStatusEngine to On.
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using [10.129.2.35](http://10.129.2.35/). Set the 'ServerName' directive globally to suppress this message
[Tue Mar 29 06:48:56.019112 2022] [ssl:warn] [pid 1:tid 139883805257152] AH01909: [10.129.2.35](http://10.129.2.35/):8443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 29 06:48:56.019250 2022] [lbmethod_heartbeat:notice] [pid 1:tid 139883805257152] AH02282: No slotmem from mod_heartmonitor
[Tue Mar 29 06:48:56.023177 2022] [mpm_event:notice] [pid 1:tid 139883805257152] AH00489: Apache/2.4.37 (Red Hat Enterprise Linux) OpenSSL/1.1.1k configured -- resuming normal operations
[Tue Mar 29 06:48:56.023195 2022] [core:notice] [pid 1:tid 139883805257152] AH00094: Command line: 'httpd -D FOREGROUND'

$ oc exec httpd-84d7895c77-6s6f5 -- ls -lR /tmp/{test1,test2}
/tmp/test1:
total 4
-rw-r--r--. 1 1001450000 root 14 Mar 29 06:48 file.data

/tmp/test2:
total 4
-rw-r--r--. 1 1001450000 root 14 Mar 29 06:48 file.data

But with the new Image, we can now also just reference a ConfigMap as volume and trigger scripts that way. With that, it's no longer required to run a source-to-image build to have the httpd-pre-init populated but instead you can just run the Image and source the scripts from the ConfigMap.

$ oc get deployment httpd-map -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "3"
  creationTimestamp: "2022-03-29T06:20:33Z"
  generation: 7
  labels:
    app: httpd-map
  name: httpd-map
  namespace: project-202
  resourceVersion: "62910805"
  uid: 5661a0ea-dee6-4930-b6ab-d1d2704e661f
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 1
  selector:
    matchLabels:
      app: httpd-map
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: httpd-map
    spec:
      containers:
      - image: quay.io/rhn_support_sreber/httpd@sha256:73b823d656a4452380e281b6b0ae688a2d021a20580f7f6ad2c4a82adc28c8db
        imagePullPolicy: IfNotPresent
        name: default
        resources:
          limits:
            cpu: 500m
            memory: 256Mi
          requests:
            cpu: 500m
            memory: 256Mi
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /opt/app-root/src/httpd-pre-init
          name: config-volume
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      serviceAccount: default
      serviceAccountName: default
      terminationGracePeriodSeconds: 5
      volumes:
      - configMap:
          defaultMode: 420
          name: httpd-scripts
        name: config-volume
status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: "2022-03-29T06:20:33Z"
    lastUpdateTime: "2022-03-29T06:43:59Z"
    message: ReplicaSet "httpd-map-fbf78c84d" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2022-03-29T06:44:10Z"
    lastUpdateTime: "2022-03-29T06:44:10Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 7
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

$ oc get cm httpd-scripts -o yaml
apiVersion: v1
data:
  script1.sh: |
    #!/bin/bash

    echo "Test 1 script start ..."

    mkdir -p /tmp/test1
    echo "Test 1 script" > /tmp/test1/file.data

    echo "Test 1 script end ..."
  script2.sh: |
    #!/bin/bash

    echo "Test 2 script start ..."

    mkdir -p /tmp/test2
    echo "Test 2 script" > /tmp/test2/file.data

    echo "Test 2 script end ..."
kind: ConfigMap
metadata:
  creationTimestamp: "2022-03-29T06:22:56Z"
  name: httpd-scripts
  namespace: project-202
  resourceVersion: "62876017"
  uid: a0abd5a9-a8e9-4098-ad6d-806c9f4862ed

The resulting pod shows again that the scripts mounted via ConfigMap are sources and executed as expected.

$ oc logs httpd-map-fbf78c84d-784n5
=> sourcing /opt/app-root/src/httpd-pre-init/..2022_03_29_06_44_03.941907033/script1.sh ...
Test 1 script start ...
Test 1 script end ...
=> sourcing /opt/app-root/src/httpd-pre-init/..2022_03_29_06_44_03.941907033/script2.sh ...
Test 2 script start ...
Test 2 script end ...
=> sourcing [10-set-mpm.sh](http://10-set-mpm.sh/) ...
=> sourcing [20-copy-config.sh](http://20-copy-config.sh/) ...
=> sourcing [40-ssl-certs.sh](http://40-ssl-certs.sh/) ...
---> Generating SSL key pair for httpd...
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using [10.130.2.197](http://10.130.2.197/). Set the 'ServerName' directive globally to suppress this message
[Tue Mar 29 06:44:10.257208 2022] [ssl:warn] [pid 1:tid 140544982937024] AH01909: [10.130.2.197](http://10.130.2.197/):8443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 29 06:44:10.257303 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity for Apache/2.9.2 (http://www.modsecurity.org/) configured.
[Tue Mar 29 06:44:10.257308 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: APR compiled version="1.6.3"; loaded version="1.6.3"
[Tue Mar 29 06:44:10.257310 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: PCRE compiled version="8.42 "; loaded version="8.42 2018-03-20"
[Tue Mar 29 06:44:10.257314 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: LUA compiled version="Lua 5.3"
[Tue Mar 29 06:44:10.257314 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: YAJL compiled version="2.1.0"
[Tue Mar 29 06:44:10.257315 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: LIBXML compiled version="2.9.7"
[Tue Mar 29 06:44:10.257317 2022] [:notice] [pid 1:tid 140544982937024] ModSecurity: Status engine is currently disabled, enable it by set SecStatusEngine to On.
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using [10.130.2.197](http://10.130.2.197/). Set the 'ServerName' directive globally to suppress this message
[Tue Mar 29 06:44:10.324359 2022] [ssl:warn] [pid 1:tid 140544982937024] AH01909: [10.130.2.197](http://10.130.2.197/):8443:0 server certificate does NOT include an ID which matches the server name
[Tue Mar 29 06:44:10.324458 2022] [lbmethod_heartbeat:notice] [pid 1:tid 140544982937024] AH02282: No slotmem from mod_heartmonitor
[Tue Mar 29 06:44:10.327214 2022] [mpm_event:notice] [pid 1:tid 140544982937024] AH00489: Apache/2.4.37 (Red Hat Enterprise Linux) OpenSSL/1.1.1k configured -- resuming normal operations
[Tue Mar 29 06:44:10.327229 2022] [core:notice] [pid 1:tid 140544982937024] AH00094: Command line: 'httpd -D FOREGROUND'

$ oc exec httpd-map-fbf78c84d-784n5 -- ls -lR /tmp/{test1,test2}
/tmp/test1:
total 4
-rw-r--r--. 1 1001450000 root 14 Mar 29 06:44 file.data

/tmp/test2:
total 4
-rw-r--r--. 1 1001450000 root 14 Mar 29 06:44 file.data

Using find -L instead of -maxdepth 10 was not working, as the ConfigMap mounts the scripts using symlinks multiple times. Therefore, with the given approach a lot of refactoring of the original Code would be required to make it work and avoid duplicated execution of the scripts.

sreber84 commented 2 years ago

cc @phracek

hhorak commented 2 years ago

[test]

phracek commented 2 years ago

@sreber84 Please rebase common submodule.

[test-all]

phracek commented 2 years ago

Rebased and closed by mistake. New pull request in favor https://github.com/sclorg/httpd-container/pull/157