search

fluent / helm-charts

Helm Charts for Fluentd and Fluent Bit
Apache License 2.0
374 stars 443 forks source link

Output plugin 'loki' cannot be loaded - fluent/fluent-bit:1.6.1 #39

Closed ghost closed 3 years ago

ghost commented 3 years ago

In our Kubernetes cluster we use Fluentbit, Loki and Grafana. Since the fluentbit image 1.6.1 contains a Loki output plugin, we wanted to use the fluentbit image 1.6.1 in the helm chart.

We get the following error message from the fluentbit pod:

2020-10-20T07:55:43.405588570Z Fluent Bit v1.6.0 2020-10-20T07:55:43.405668159Z * Copyright (C) 2019-2020 The Fluent Bit Authors 2020-10-20T07:55:43.405677844Z * Copyright (C) 2015-2018 Treasure Data 2020-10-20T07:55:43.405683483Z * Fluent Bit is a CNCF sub-project under the umbrella of Fluentd 2020-10-20T07:55:43.405688511Z * https://fluentbit.io 2020-10-20T07:55:43.405693483Z 2020-10-20T07:55:43.406466289Z Output plugin 'loki' cannot be loaded 2020-10-20T07:55:43.406488412Z Error: You must specify an output target. Aborting 2020-10-20T07:55:43.406494765Z

Our values.yml looks like this:


kind: DaemonSet
replicaCount: 1
image:
  repository: "fluent/fluent-bit"
  pullPolicy: Always
  tag: "1.6.1"

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""
serviceAccount:
  create: true
  annotations: {}
  name:
rbac:
  create: true
podSecurityPolicy:
  create: false
podSecurityContext: {}
securityContext: {}
service:
  type: ClusterIP
  port: 2020
  labels: {}
  annotations: {}
serviceMonitor:
  enabled: false
resources: {}
nodeSelector: {}
tolerations: []
affinity: {}
podAnnotations: {}
podLabels: {}
priorityClassName: ""
env: []
envFrom: []
extraPorts: []
extraVolumes: []
extraVolumeMounts: []
config:
  service: |
    [SERVICE]
        Flush 1
        Daemon Off
        Log_Level info
        Parsers_File parsers.conf
        Parsers_File custom_parsers.conf
        HTTP_Server On
        HTTP_Listen 0.0.0.0
        HTTP_Port 2020
  inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/*.log
        Parser docker
        Tag kube.*
        Mem_Buf_Limit 10MB
        Refresh_Interval  10
        Ignore_Older 2d
        Skip_Long_Lines On
  filters: |
    [FILTER]
        Name kubernetes
        Match kube.*
        Merge_Log On
        Merge_Log_Key app
        Keep_Log Off
        K8S-Logging.Parser Off
        K8S-Logging.Exclude Off
  outputs: |
    [OUTPUT]
        name               loki
        match              *
        host               loki.monitoring.svc.cluster.local
        port               3100
  customParsers: |
    [PARSER]
        Name docker_no_time
        Format json
        Time_Keep Off
        Time_Key time
        Time_Format %Y-%m-%dT%H:%M:%S.%L

I don't quite understand why the log fluentbit says version 1.6.0 although we specified version 1.6.1. In the pod description there is also version 1.6.1. I suspect that this is why it does not recognize the output loki plugin.

ghost commented 3 years ago

https://github.com/fluent/fluent-bit/issues/2707

omidraha commented 11 months ago

What's the valid value for host in the loki config section?

host               loki.monitoring.svc.cluster.local

I have issue https://github.com/fluent/helm-charts/issues/415 with configuration settings.

kubectl get services  -A
NAMESPACE         NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                        AGE
default           kubelet                       ClusterIP   None             <none>        10250/TCP,10255/TCP,4194/TCP   46m
default           kubernetes                    ClusterIP   10.100.0.1       <none>        443/TCP                        5h
fluent-bit        fluent-bit-aab8a10b           ClusterIP   10.100.187.178   <none>        2020/TCP                       72m
fluent-bit        loki-backend                  ClusterIP   10.100.246.54    <none>        3100/TCP,9095/TCP              22m
fluent-bit        loki-backend-headless         ClusterIP   None             <none>        3100/TCP,9095/TCP              22m
fluent-bit        loki-canary                   ClusterIP   10.100.59.198    <none>        3500/TCP                       22m
fluent-bit        loki-gateway                  ClusterIP   10.100.7.236     <none>        80/TCP                         22m
fluent-bit        loki-memberlist               ClusterIP   None             <none>        7946/TCP                       22m
fluent-bit        loki-read                     ClusterIP   10.100.41.44     <none>        3100/TCP,9095/TCP              22m
fluent-bit        loki-read-headless            ClusterIP   None             <none>        3100/TCP,9095/TCP              22m
fluent-bit        loki-write                    ClusterIP   10.100.148.17    <none>        3100/TCP,9095/TCP              22m
fluent-bit        loki-write-headless           ClusterIP   None             <none>        3100/TCP,9095/TCP              22m
fluent-bit        query-scheduler-discovery     ClusterIP   None             <none>        3100/TCP,9095/TCP              22m