[stable/graylog] How to mount a file inside graylog container using helm chart

[GaneshbabuRamamoorthy]

Hi All,

I am using this helm chart Graylog for analysis of log data. I am trying to mount the file inside the Graylog container (mount_path=/usr/share/graylog/plugin) after executing the helm install command, right now I am manually doing kubectl cp command to move the file inside the container.

In the helm charts graylog documentation they have given the option to include files

below is the way I have tried to include the file in graylog container,

  ## Additional server files will be deployed to /etc/graylog/server
  ## For example, you can put server certificates or authorized clients certificates here
  ##
  serverFiles:
    graylog-pubsub-key.json: |
      {
        "type": "service_account",
        "project_id": "gcp",
        "private_key_id": "skdsf3234nspkew23",
        "private_key": "-----BEGIN PRIVATE KEY-----\test=\n-----END PRIVATE KEY-----\n",
        "client_email": "external-publish-test@gcp.iam.gserviceaccount.com",
        "client_id": "6453493934",
        "auth_uri": "https://accounts.google.com/o/oauth2/auth",
        "token_uri": "https://oauth2.googleapis.com/token",
        "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
        "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/external-publish-test%40gcp.iam.gserviceaccount.com"
      }  

  tls:
    enabled: true
    keyFile: /usr/share/graylog/plugin/graylog-pubsub-key.json

Other option I have tried by using extraInitContainers in the value.yaml file, but still it is unable to mount the JSON file inside the container,

extraInitContainers:

  - name: init-plugins
    image: "appropriate/curl:latest"
    imagePullPolicy: "IfNotPresent"
    volumeMounts:
      - name: plugin
        mountPath: /usr/share/graylog/plugin
    command:
      - /bin/sh
      - -c
      - |
        cat <<'EOF' > graylog-pubsub-key.json
        {
         "type": "service_account",
         "project_id": "gcp",
         "private_key_id": "skdsf3234nspkew23",
         "private_key": "-----BEGIN PRIVATE KEY-----\test=\n-----END PRIVATE KEY-----\n",
         "client_email": "external-publish-test@gcp.iam.gserviceaccount.com",
         "client_id": "6453493934",
         "auth_uri": "https://a
ccounts.google.com/o/oauth2/auth",
         "token_uri": "https://oauth2.googleapis.com/token",
         "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
         "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/external-publish-test%40gcp.iam.gserviceaccount.com"
         } 
         EOF
         cp graylog-pubsub-key.json /plugin/

Please correct me if I am doing anything wrong in the configuration. Attached the graylog helm value.yaml which I tried in my local environment.

# Default values for Graylog.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

rbac:
  create: true

serviceAccount:
  create: true
  name: graylog-sa

tags:
  install-elasticsearch: false
  install-mongodb: false

graylog:
  image:
    repository: "graylog/graylog:3.1.3-2"
    pullPolicy: "IfNotPresent"

  replicas: 2

  env:
    GRAYLOG_HTTP_EXTERNAL_URI: http://gcp-graylog:9000/

  affinity: {}
  tolerations: []

  extraInitContainers:
  ## init containers to run d: "appropriate/curl:latest graylog/graylog:3.1.3-2"
    - name: setup
      image: "appropriate/curl:latest"
      imagePullPolicy: "IfNotPresent"
      volumeMounts:
        - name: plugin
          mountPath: /usr/share/graylog/plugin
      command:
        - /bin/sh
        - -c
        - |
          cat <<'EOF' > graylog-pubsub-key.json
          {
           "type": "service_account",
           "project_id": "gcp",
           "private_key_id": "skdsf3234nspkew23",
           "private_key": "-----BEGIN PRIVATE KEY-----\test=\n-----END PRIVATE KEY-----\n",
           "client_email": "external-publish-test@gcp.iam.gserviceaccount.com",
           "client_id": "6453493934",
           "auth_uri": "https://accounts.google.com/o/oauth2/auth",
           "token_uri": "https://oauth2.googleapis.com/token",
           "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
           "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/external-publish-test%40gcp.iam.gserviceaccount.com"
          }  
          EOF
          cp graylog-pubsub-key.json /plugin/

  nodeSelector: {}

  podAnnotations: {}

  persistence:

    enabled: false
    accessMode: ReadWriteOnce
    size: "20Gi"
  plugins:   

  service:
    type: ClusterIP
    port: 9000
    master:
      annotations: {}
      port: 9000

  input: 
    tcp:
      service:
        type: ClusterIP
      ports:
        - name: gelf
          port: 12201
        - name: beats
          port: 5044
    udp:
      service:
        type: ClusterIP
      ports:
        - name: syslog
          port: 12222

  ingress:

    enabled: false

    annotations: {}
    hosts: []
    tls: []
  resources:
    limits:
      cpu: "2"
      memory: "7Gi"
    requests:
      cpu: "1"
      memory: "1Gi"

  heapSize: "4g"

  updateStrategy: OnDelete
  terminationGracePeriodSeconds: 120
  metrics:
    enabled: true

  geoip:
    enabled: false
  rootUsername: "admin"
  rootPassword: "auto123"
  rootEmail: ""
  rootTimezone: "UTC"

  elasticsearch:
    hosts: http://elasticsearch-master-headless:9200,http://elasticsearch-data-headless:9200

  mongodb:
    uri: "mongodb://graylog-mongodb-mongodb-replicaset:27017/graylogedge"
    maxConnections: 1000

  transportEmail:
    enabled: false
    hostname: ""
    port: 2587
    useAuth: true
    useTls: true
    useSsl: false
    authUsername: ""
    authPassword: ""
    subjectPrefix: "[graylog]"
    fromEmail: ""

  config: |
     is_master = true
     elasticsearch_index_prefix = graylog3

  journal:
    deleteBeforeStart: true

  serverFiles: {}

  tls:
    enabled: false

  provisioner:
    enabled: false

## Specify Elasticsearch version from requirement dependencies. Ignore this seection if you install Elasticsearch manually.
## Note: Graylog 2.4 requires Elasticsearch version <= 5.6
elasticsearch:
  image:
    repository: "docker.elastic.co/elasticsearch/elasticsearch-oss"
    tag: "6.5.4"
  cluster:
    xpackEnable: false

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[StefanLiebenberg]

I had some success using config maps:

kubectl create configmap graylog-content-packs --from-file=graylog/content-packs/

graylog:
  env:
      GRAYLOG_CONTENT_PACKS_LOADER_ENABLED: true
      GRAYLOG_CONTENT_PACKS_DIR: /usr/share/graylog/content-packs
      GRAYLOG_CONTENT_PACKS_AUTO_INSTALL: MyContentPack
  extraVolumes:
   - name: graylog-content-packs
     configMap:
       name: graylog-content-packs
  extraVolumeMounts:
  - name: graylog-content-packs
    mountPath: /usr/share/graylog/content-packs

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Any further update will cause the issue/pull request to no longer be considered stale. Thank you for your contributions.

[stale[bot]]

This issue is being automatically closed due to inactivity.