search

fluent / fluent-bit

Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows
https://fluentbit.io
Apache License 2.0
5.84k stars 1.58k forks source link

tail input plugin /var/log/containers/*.log doesnt push logs to destination #8213

Closed tanmaya-panda1 closed 8 months ago

tanmaya-panda1 commented 11 months ago

Bug Report

Describe the bug I am trying to push kubernetes logs to kusto as destination. tail input plugin /var/log/containers/*.log doesn't seem to work but it works when i use /var/log/containers/fluent*.log. I have added the volume mounts as well to avoid the symlinks issue.

I am using helm chart with the following config

helm repo add fluent https://fluent.github.io/helmcharts

helm repo update helm install fluent-bit-demo fluent/fluent-bit --values kusto-values-template.yaml

The contents of kusto-values-template.yaml is as follows

logLevel: trace

kind: DaemonSet

image:
  repository: fluentbitcontainereast.azurecr.io/fluent-bit-kusto
  tag: 2.1.9
  pullPolicy: Always

config:
  service: |
    [SERVICE]
        Daemon Off
        Flush 1
        Log_Level trace
        HTTP_Server On
        HTTP_Listen 0.0.0.0
        Parsers_File parsers.conf
        Parsers_File custom_parsers.conf
        HTTP_Port 2020
        Health_Check On

  inputs: |
    [INPUT]
        Name tail
        Path /var/log/containers/`*.log`
        multiline.parser docker, cri
        Tag kube.*
        Mem_Buf_Limit 1MB
        Skip_Long_Lines Off
        Refresh_Interval 10

  filters: |
    [FILTER]
        Name kubernetes
        Match *
        Merge_Log On
        Merge_Log_key log_processed
        K8S-Logging.Parser On
        K8S-Logging.Exclude Off

  outputs: |
    [OUTPUT]
        name azure_kusto
        match *
        tenant_id xxxxx
        client_id xxxxx
        client_secret xxxxx
        ingestion_endpoint xxxxx
        database_name xxxx
        table_name xxxx
        ingestion_mapping_reference xxxx

  customParsers: |
    [PARSER]
        Name docker_no_time
        Format json
        Time_Keep On
        Time_Key time
        Time_Format %Y-%m-%dT%H:%M:%S.%L

volumeMounts:
  - name: config
    mountPath: /fluent-bit/etc/conf
  - name: var-log-containers
    mountPath: /var/log/containers
  - name: var-log-pods
    mountPath: /var/log/pods
  - name: var-lib-docker-containers
    mountPath: /var/lib/docker/containers     

daemonSetVolumes:
  - name: varlog
    hostPath:
      path: /var/log/containers
  - name: varpods
    hostPath:
      path: /var/log/pods   
  - name: varlibdockercontainers
    hostPath:
      path: /var/lib/docker/containers
  - name: etcmachineid
    hostPath:
      path: /etc/machine-id
      type: File

daemonSetVolumeMounts:
  - name: varlog
    mountPath: /var/log/containers
  - name: varpods
    mountPath: /var/log/pods 
  - name: varlibdockercontainers
    mountPath: /var/lib/docker/containers
    readOnly: true
  - name: etcmachineid
    mountPath: /etc/machine-id
    readOnly: true

To Reproduce

[2023/11/24 21:21:27] [ info] Configuration: [2023/11/24 21:21:27] [ info] flush time | 1.000000 seconds [2023/11/24 21:21:27] [ info] grace | 5 seconds [2023/11/24 21:21:27] [ info] daemon | 0 [2023/11/24 21:21:27] [ info] [2023/11/24 21:21:27] [ info] inputs: [2023/11/24 21:21:27] [ info] tail [2023/11/24 21:21:27] [ info] [2023/11/24 21:21:27] [ info] filters: [2023/11/24 21:21:27] [ info] kubernetes.0 [2023/11/24 21:21:27] [ info] ___ [2023/11/24 21:21:27] [ info] outputs: [2023/11/24 21:21:27] [ info] azure_kusto.0 [2023/11/24 21:21:27] [ info] ___ [2023/11/24 21:21:27] [ info] collectors: [2023/11/24 21:21:27] [ info] [fluent bit] version=2.2.1, commit=8733b25df1, pid=1 [2023/11/24 21:21:27] [debug] [engine] coroutine stack size: 24576 bytes (24.0K) [2023/11/24 21:21:27] [ info] [storage] ver=1.5.1, type=memory, sync=normal, checksum=off, max_chunks_up=128 [2023/11/24 21:21:27] [ info] [cmetrics] version=0.6.5 [2023/11/24 21:21:27] [ info] [ctraces ] version=0.3.1 [2023/11/24 21:21:27] [ info] [input:tail:tail.0] initializing [2023/11/24 21:21:27] [ info] [input:tail:tail.0] storage_strategy='memory' (memory only) [2023/11/24 21:21:27] [debug] [tail:tail.0] created event channels: read=21 write=22 [2023/11/24 21:21:27] [ info] [input:tail:tail.0] multiline core started [2023/11/24 21:21:27] [debug] [input:tail:tail.0] flb_tail_fs_inotify_init() initializing inotify tail input [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inotify watch fd=28 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scanning path /var/log/containers/*.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1813411 with offset=386852 appended as /var/log/containers/ama-logs-d668n_kube-system_addon-token-adapter-322f5b5f1cbb9a10f662da26ca34c3a29047b0947b5bda87ef01a7990eb43eb4.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/ama-logs-d668n_kube-system_addon-token-adapter-322f5b5f1cbb9a10f662da26ca34c3a29047b0947b5bda87ef01a7990eb43eb4.log, inode 1813411 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1813459 with offset=8861 appended as /var/log/containers/ama-logs-d668n_kube-system_ama-logs-bf6949ab252b9ceda2d9023aa1c293535c87e592f5022a886d5fd52f0af72eb5.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/ama-logs-d668n_kube-system_ama-logs-bf6949ab252b9ceda2d9023aa1c293535c87e592f5022a886d5fd52f0af72eb5.log, inode 1813459 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1813497 with offset=4591 appended as /var/log/containers/ama-logs-d668n_kube-system_ama-logs-prometheus-75d72411158111544c959d11394a9c804495b7c3617a4eb0db39e1bdba9b8b44.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/ama-logs-d668n_kube-system_ama-logs-prometheus-75d72411158111544c959d11394a9c804495b7c3617a4eb0db39e1bdba9b8b44.log, inode 1813497 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1813431 with offset=337024 appended as /var/log/containers/ama-metrics-node-tvs5r_kube-system_addon-token-adapter-83fab8b75208fab94ca166bbb4a154b83734be35521c609334722d60d72c2fdd.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/ama-metrics-node-tvs5r_kube-system_addon-token-adapter-83fab8b75208fab94ca166bbb4a154b83734be35521c609334722d60d72c2fdd.log, inode 1813431 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1810243 with offset=9808 appended as /var/log/containers/ama-metrics-node-tvs5r_kube-system_prometheus-collector-e3de35fc24f6e5e7b0f50a87468521f98a5e327caf5ed91f71a8f452884c1b5e.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/ama-metrics-node-tvs5r_kube-system_prometheus-collector-e3de35fc24f6e5e7b0f50a87468521f98a5e327caf5ed91f71a8f452884c1b5e.log, inode 1810243 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1806932 with offset=4268058 appended as /var/log/containers/azure-ip-masq-agent-89vgs_kube-system_azure-ip-masq-agent-81675abb5b05bf9588ba028c9641c051e2f61c1342317358ac8ab46988e2df0e.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/azure-ip-masq-agent-89vgs_kube-system_azure-ip-masq-agent-81675abb5b05bf9588ba028c9641c051e2f61c1342317358ac8ab46988e2df0e.log, inode 1806932 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1814633 with offset=74955 appended as /var/log/containers/calico-kube-controllers-7957ff69b4-j4r28_calico-system_calico-kube-controllers-100385987bdaf0dad011964030ec8018f65f625b08af1e788a35500b75144125.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/calico-kube-controllers-7957ff69b4-j4r28_calico-system_calico-kube-controllers-100385987bdaf0dad011964030ec8018f65f625b08af1e788a35500b75144125.log, inode 1814633 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807538 with offset=3164510 appended as /var/log/containers/calico-node-56c7v_calico-system_calico-node-557fd3750d680c5b05b44cdc5d3ebf4972aeeeddd18c79744d805e13eaa3ec45.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/calico-node-56c7v_calico-system_calico-node-557fd3750d680c5b05b44cdc5d3ebf4972aeeeddd18c79744d805e13eaa3ec45.log, inode 1807538 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807037 with offset=0 appended as /var/log/containers/calico-node-56c7v_calico-system_flexvol-driver-54be58e8799a9af362ce04017e36a19b3e31f70e98a04a4ca25ca691c9d3b775.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/calico-node-56c7v_calico-system_flexvol-driver-54be58e8799a9af362ce04017e36a19b3e31f70e98a04a4ca25ca691c9d3b775.log, inode 1807037 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807087 with offset=1003 appended as /var/log/containers/cloud-node-manager-6p8nd_kube-system_cloud-node-manager-90eec4734379b7f22a84964060b74991a16abd8e88ba62319d19aa7ffc03be25.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/cloud-node-manager-6p8nd_kube-system_cloud-node-manager-90eec4734379b7f22a84964060b74991a16abd8e88ba62319d19aa7ffc03be25.log, inode 1807087 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807261 with offset=12043 appended as /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_azuredisk-ac97724d4596642ef35c1d2d0c576568312baf10786a6b9fda40e7eed9edcbb7.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_azuredisk-ac97724d4596642ef35c1d2d0c576568312baf10786a6b9fda40e7eed9edcbb7.log, inode 1807261 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807180 with offset=368 appended as /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_liveness-probe-0053c18fa4754385a806000f15ceab7f2ee64c167b06d2a182161cfb5ae7deb5.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_liveness-probe-0053c18fa4754385a806000f15ceab7f2ee64c167b06d2a182161cfb5ae7deb5.log, inode 1807180 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807212 with offset=1586 appended as /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_node-driver-registrar-d704be2678e43eb22ca8b4cca4dfe65e010b5ef73055cd6a1d83a63d0686f8d6.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azuredisk-node-xhtjv_kube-system_node-driver-registrar-d704be2678e43eb22ca8b4cca4dfe65e010b5ef73055cd6a1d83a63d0686f8d6.log, inode 1807212 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807264 with offset=12609 appended as /var/log/containers/csi-azurefile-node-kfxkd_kube-system_azurefile-da1ed79280215f04afef0531afcb04a61ab3c466fe2dda6725cef7b7b8844293.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azurefile-node-kfxkd_kube-system_azurefile-da1ed79280215f04afef0531afcb04a61ab3c466fe2dda6725cef7b7b8844293.log, inode 1807264 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807138 with offset=368 appended as /var/log/containers/csi-azurefile-node-kfxkd_kube-system_liveness-probe-e960a07123f3ce8e9b5625832ee21c416bb84b3ba87c588015fce18ba3b04054.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azurefile-node-kfxkd_kube-system_liveness-probe-e960a07123f3ce8e9b5625832ee21c416bb84b3ba87c588015fce18ba3b04054.log, inode 1807138 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807211 with offset=1584 appended as /var/log/containers/csi-azurefile-node-kfxkd_kube-system_node-driver-registrar-db225223c3baee655dcfb492ea14e85caace41e209fa32c364d2c941023bc964.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/csi-azurefile-node-kfxkd_kube-system_node-driver-registrar-db225223c3baee655dcfb492ea14e85caace41e209fa32c364d2c941023bc964.log, inode 1807211 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=4128853 with offset=11142 appended as /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, inode 4128853 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1807489 with offset=1127548 appended as /var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-6d86c3689c16d8c7a9bdff2e42982f8f9bed657303457593fcf5ae58ab581b4e.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-6d86c3689c16d8c7a9bdff2e42982f8f9bed657303457593fcf5ae58ab581b4e.log, inode 1807489 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] inode=1806977 with offset=299 appended as /var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-bootstrap-b6a0314e1611fbe649d960720cd93f2742362eb2241d9241605902dbe0537d0b.log [2023/11/24 21:21:27] [debug] [input:tail:tail.0] scan_glob add(): /var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-bootstrap-b6a0314e1611fbe649d960720cd93f2742362eb2241d9241605902dbe0537d0b.log, inode 1806977 [2023/11/24 21:21:27] [debug] [input:tail:tail.0] 19 new files found on path '/var/log/containers/*.log' [2023/11/24 21:21:27] [ info] [filter:kubernetes:kubernetes.0] https=1 host=kubernetes.default.svc port=443 [2023/11/24 21:21:27] [ info] [filter:kubernetes:kubernetes.0] token updated [2023/11/24 21:21:27] [ info] [filter:kubernetes:kubernetes.0] local POD info OK [2023/11/24 21:21:32] [ info] [filter:kubernetes:kubernetes.0] testing connectivity with API server... [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Send out request to API Server for pods information [2023/11/24 21:21:32] [debug] [http_client] not using http_proxy for header [2023/11/24 21:21:32] [debug] [http_client] server kubernetes.default.svc:443 will close connection #48 [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Request (ns=default, pod=fluent-bit-demo-r9bj5) http_do=0, HTTP Status: 200 [2023/11/24 21:21:32] [ info] [filter:kubernetes:kubernetes.0] connectivity OK [2023/11/24 21:21:32] [debug] [azure_kusto:azure_kusto.0] created event channels: read=48 write=49 [2023/11/24 21:21:32] [ info] [output:azure_kusto:azure_kusto.0] endpoint='https://ingest-sdkse2etest.eastus.kusto.windows.net', database='e2e', table='FluentBitTemp' [2023/11/24 21:21:32] [ info] [http_server] listen iface=0.0.0.0 tcp_port=2020 [2023/11/24 21:21:32] [ info] [sp] stream processor started [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1813411 file=/var/log/containers/ama-logs-d668n_kube-system_addon-token-adapter-322f5b5f1cbb9a10f662da26ca34c3a29047b0947b5bda87ef01a7990eb43eb4.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1813411 watch_fd=1 name=/var/log/containers/ama-logs-d668n_kube-system_addon-token-adapter-322f5b5f1cbb9a10f662da26ca34c3a29047b0947b5bda87ef01a7990eb43eb4.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1813459 file=/var/log/containers/ama-logs-d668n_kube-system_ama-logs-bf6949ab252b9ceda2d9023aa1c293535c87e592f5022a886d5fd52f0af72eb5.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1813459 watch_fd=2 name=/var/log/containers/ama-logs-d668n_kube-system_ama-logs-bf6949ab252b9ceda2d9023aa1c293535c87e592f5022a886d5fd52f0af72eb5.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1813497 file=/var/log/containers/ama-logs-d668n_kube-system_ama-logs-prometheus-75d72411158111544c959d11394a9c804495b7c3617a4eb0db39e1bdba9b8b44.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1813497 watch_fd=3 name=/var/log/containers/ama-logs-d668n_kube-system_ama-logs-prometheus-75d72411158111544c959d11394a9c804495b7c3617a4eb0db39e1bdba9b8b44.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1813431 file=/var/log/containers/ama-metrics-node-tvs5r_kube-system_addon-token-adapter-83fab8b75208fab94ca166bbb4a154b83734be35521c609334722d60d72c2fdd.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1813431 watch_fd=4 name=/var/log/containers/ama-metrics-node-tvs5r_kube-system_addon-token-adapter-83fab8b75208fab94ca166bbb4a154b83734be35521c609334722d60d72c2fdd.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1810243 file=/var/log/containers/ama-metrics-node-tvs5r_kube-system_prometheus-collector-e3de35fc24f6e5e7b0f50a87468521f98a5e327caf5ed91f71a8f452884c1b5e.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1810243 watch_fd=5 name=/var/log/containers/ama-metrics-node-tvs5r_kube-system_prometheus-collector-e3de35fc24f6e5e7b0f50a87468521f98a5e327caf5ed91f71a8f452884c1b5e.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1806932 file=/var/log/containers/azure-ip-masq-agent-89vgs_kube-system_azure-ip-masq-agent-81675abb5b05bf9588ba028c9641c051e2f61c1342317358ac8ab46988e2df0e.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1806932 watch_fd=6 name=/var/log/containers/azure-ip-masq-agent-89vgs_kube-system_azure-ip-masq-agent-81675abb5b05bf9588ba028c9641c051e2f61c1342317358ac8ab46988e2df0e.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1814633 file=/var/log/containers/calico-kube-controllers-7957ff69b4-j4r28_calico-system_calico-kube-controllers-100385987bdaf0dad011964030ec8018f65f625b08af1e788a35500b75144125.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1814633 watch_fd=7 name=/var/log/containers/calico-kube-controllers-7957ff69b4-j4r28_calico-system_calico-kube-controllers-100385987bdaf0dad011964030ec8018f65f625b08af1e788a35500b75144125.log [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Send out request to API Server for pods information [2023/11/24 21:21:32] [debug] [http_client] not using http_proxy for header [2023/11/24 21:21:32] [debug] [http_client] server kubernetes.default.svc:443 will close connection #71 [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Request (ns=calico-system, pod=calico-node-56c7v) http_do=0, HTTP Status: 200 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=33104, records=34, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807037 file=/var/log/containers/calico-node-56c7v_calico-system_flexvol-driver-54be58e8799a9af362ce04017e36a19b3e31f70e98a04a4ca25ca691c9d3b775.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807037 watch_fd=8 name=/var/log/containers/calico-node-56c7v_calico-system_flexvol-driver-54be58e8799a9af362ce04017e36a19b3e31f70e98a04a4ca25ca691c9d3b775.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807087 file=/var/log/containers/cloud-node-manager-6p8nd_kube-system_cloud-node-manager-90eec4734379b7f22a84964060b74991a16abd8e88ba62319d19aa7ffc03be25.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807087 watch_fd=9 name=/var/log/containers/cloud-node-manager-6p8nd_kube-system_cloud-node-manager-90eec4734379b7f22a84964060b74991a16abd8e88ba62319d19aa7ffc03be25.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807261 file=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_azuredisk-ac97724d4596642ef35c1d2d0c576568312baf10786a6b9fda40e7eed9edcbb7.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807261 watch_fd=10 name=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_azuredisk-ac97724d4596642ef35c1d2d0c576568312baf10786a6b9fda40e7eed9edcbb7.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807180 file=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_liveness-probe-0053c18fa4754385a806000f15ceab7f2ee64c167b06d2a182161cfb5ae7deb5.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807180 watch_fd=11 name=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_liveness-probe-0053c18fa4754385a806000f15ceab7f2ee64c167b06d2a182161cfb5ae7deb5.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807212 file=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_node-driver-registrar-d704be2678e43eb22ca8b4cca4dfe65e010b5ef73055cd6a1d83a63d0686f8d6.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807212 watch_fd=12 name=/var/log/containers/csi-azuredisk-node-xhtjv_kube-system_node-driver-registrar-d704be2678e43eb22ca8b4cca4dfe65e010b5ef73055cd6a1d83a63d0686f8d6.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807264 file=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_azurefile-da1ed79280215f04afef0531afcb04a61ab3c466fe2dda6725cef7b7b8844293.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807264 watch_fd=13 name=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_azurefile-da1ed79280215f04afef0531afcb04a61ab3c466fe2dda6725cef7b7b8844293.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807138 file=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_liveness-probe-e960a07123f3ce8e9b5625832ee21c416bb84b3ba87c588015fce18ba3b04054.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807138 watch_fd=14 name=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_liveness-probe-e960a07123f3ce8e9b5625832ee21c416bb84b3ba87c588015fce18ba3b04054.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807211 file=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_node-driver-registrar-db225223c3baee655dcfb492ea14e85caace41e209fa32c364d2c941023bc964.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807211 watch_fd=15 name=/var/log/containers/csi-azurefile-node-kfxkd_kube-system_node-driver-registrar-db225223c3baee655dcfb492ea14e85caace41e209fa32c364d2c941023bc964.log [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Send out request to API Server for pods information [2023/11/24 21:21:32] [debug] [http_client] not using http_proxy for header [2023/11/24 21:21:32] [debug] [http_client] server kubernetes.default.svc:443 will close connection #71 [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Request (ns=default, pod=fluent-bit-demo-r9bj5) http_do=0, HTTP Status: 200 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=53110, records=54, input=tail.0 [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Send out request to API Server for pods information [2023/11/24 21:21:32] [debug] [http_client] not using http_proxy for header [2023/11/24 21:21:32] [debug] [http_client] server kubernetes.default.svc:443 will close connection #71 [2023/11/24 21:21:32] [debug] [filter:kubernetes:kubernetes.0] Request (ns=kube-system, pod=kube-proxy-ssrlq) http_do=0, HTTP Status: 200 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=8943, records=11, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1806977 file=/var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-bootstrap-b6a0314e1611fbe649d960720cd93f2742362eb2241d9241605902dbe0537d0b.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1806977 watch_fd=16 name=/var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-bootstrap-b6a0314e1611fbe649d960720cd93f2742362eb2241d9241605902dbe0537d0b.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 21.3K [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807538 file=/var/log/containers/calico-node-56c7v_calico-system_calico-node-557fd3750d680c5b05b44cdc5d3ebf4972aeeeddd18c79744d805e13eaa3ec45.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807538 watch_fd=17 name=/var/log/containers/calico-node-56c7v_calico-system_calico-node-557fd3750d680c5b05b44cdc5d3ebf4972aeeeddd18c79744d805e13eaa3ec45.log [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=15123, records=16, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=1807489 file=/var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-6d86c3689c16d8c7a9bdff2e42982f8f9bed657303457593fcf5ae58ab581b4e.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=1807489 watch_fd=18 name=/var/log/containers/kube-proxy-ssrlq_kube-system_kube-proxy-6d86c3689c16d8c7a9bdff2e42982f8f9bed657303457593fcf5ae58ab581b4e.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 3.0K [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=2048, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 536b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=3837, records=4, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 813b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1794, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 282b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1794, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 282b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1792, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 280b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1794, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 282b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1795, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 283b [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853 file=/var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log promote to TAIL_EVENT [2023/11/24 21:21:32] [ info] [input:tail:tail.0] inotify_fs_add(): inode=4128853 watch_fd=19 name=/var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log [2023/11/24 21:21:32] [debug] [input:tail:tail.0] [static files] processed 0b, done [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=4607, records=5, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=2915, records=3, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=917, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=2845, records=3, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1927, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1928, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1929, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=1930, records=2, input=tail.0 [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=918, records=1, input=tail.0 [2023/11/24 21:21:32] [debug] [input:tail:tail.0] inode=4128853, /var/log/containers/fluent-bit-demo-r9bj5_default_fluent-bit-52c68d0654561b70061e6467316cb7d418bdd3674cfb29741bcebd732303ff3f.log, events: IN_MODIFY [2023/11/24 21:21:32] [debug] [input chunk] update output instances with new chunk size diff=919, records=1, input=tail.0 [2023/11/24 21:21:32] [ warn] [input] tail.0 paused (mem buf overlimit) [2023/11/24 21:21:32] [ info] [input] pausing tail.0 [2023/11/24 21:21:33] [debug] [task] created task=0x7f3659a3ac40 id=0 OK [2023/11/24 21:21:33] [debug] [task] created task=0x7f3659a3ad80 id=1 OK [2023/11/24 21:21:33] [debug] [task] created task=0x7f3659a3aec0 id=2 OK [2023/11/24 21:21:33] [trace] [output:azure_kusto:azure_kusto.0 at /src/fluent-bit/plugins/out_azure_kusto/azure_kusto.c:374] flushing bytes 33104 [2023/11/24 21:21:33] [ info] [output:azure_kusto:azure_kusto.0] loading kusto ingestion resourcs [2023/11/24 21:21:33] [trace] [output:azure_kusto:azure_kusto.0 at /src/fluent-bit/plugins/out_azure_kusto/azure_kusto.c:374] flushing bytes 958237


- Steps to reproduce the problem:

**Expected behavior**
<!--- A clear and concise description of what you expected to happen. -->

**Screenshots**
<!--- If applicable, add screenshots to help explain your problem. -->

**Your Environment**
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used:
* Configuration:
* Environment name and version (e.g. Kubernetes? What version?):
* Server type and version:
* Operating System and version:
* Filters and plugins:

**Additional context**
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->
tanmaya-panda1 commented 11 months ago

complete-log.txt

github-actions[bot] commented 8 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] commented 8 months ago

This issue was closed because it has been stalled for 5 days with no activity.