CPU Continuously growing with Fluent-bit version > 2.0.14 on Windows Server 2019 with Multiline Filter Plugin

[ganga1980]

Bug Report

Describe the bug CPU Continuously growing with Fluent-bit version > 2.0.14 on Windows Server 2019 with Multiline Filter Plugin. This is not issue with Fluent-bit version 2.0.14. Tried all the versions 2.1. and 2.2., and all these versions has same issue.

To Reproduce

• Rubular link if applicable:

• Example log message if applicable:

  
  apiVersion: apps/v1
  kind: DaemonSet
  metadata:
  name: fluent-bit-windows
  labels:
  app: fluent-bit-windows
  spec:
  selector:
  matchLabels:
    app: fluent-bit-windows
  template:
  metadata:
    labels:
      app: fluent-bit-windows
  spec:
    affinity:
     nodeAffinity:
       requiredDuringSchedulingIgnoredDuringExecution:
         nodeSelectorTerms:
         - matchExpressions:
           - key: kubernetes.io/os
             operator: In
             values:
             - windows
           - key: kubernetes.azure.com/os-sku
             operator: In
             values:
             - Windows2019 # For Windows 2022, use Windows2022
    containers:
    - name: fluent-bit
      image: fluent/fluent-bit:windows-2019-2.2.2 # For Windows 2022, use 2022 instead of 2019
      imagePullPolicy: IfNotPresent
      resources:
        limits:
          memory: "750Mi"
          cpu: "500m"
        # requests:
        #   memory: "100Mi"
        #   cpu: "50m"
      volumeMounts:
      - mountPath: C:\ProgramData\docker\containers
        name: docker-windows-containers
        readOnly: true
      - mountPath: C:\var #Read + Write access on this for position file
        name: docker-windows-kuberenetes-container-logs
      - name: config-volume
        mountPath: /fluent-bit/etc/
    volumes:
    - name: docker-windows-kuberenetes-container-logs
      hostPath:
        path: C:\var
    - name: docker-windows-containers
      hostPath:
        path: C:\ProgramData\docker\containers
        type: DirectoryOrCreate
    - name: config-volume
      configMap:
        name: fluent-bit-config
  ---
  apiVersion: v1
  kind: ConfigMap
  metadata:
  name: fluent-bit-config
  data:
  fluent-bit.conf: |
  [SERVICE]
      Flush         1
      Log_Level     info
      Daemon        off
      Parsers_File  parsers.conf
  
  [INPUT]
      Name                tail
      Path                C:\\var\\log\\containers\*.log
      Multiline.Parser    cri
      Tag                 windows.*
      Read_from_Head      true
      DB                  C:\\var\\log\\fblogs.db
      DB.Sync             Off
      Rotate_Wait         20
      Refresh_Interval    30
      Path_Key            filepath
      Skip_Long_Lines     On
  
  [FILTER]
      Name multiline
      Match *
      multiline.key_content log
      multiline.parser go,java,python
  
  [OUTPUT]
      Name          Null
      Match         *

- Steps to reproduce the problem:
1. Apply this yaml on to Kubernetes cluster with windows server 2019 nodes
kubectl apply -f https://raw.githubusercontent.com/microsoft/Docker-Provider/gangams/fix-windows-agent-cpu-breaches/test/fluent-bit-windows.yaml
2. Deploy some test workloads for example
kubectl apply -f https://raw.githubusercontent.com/microsoft/Docker-Provider/gangams/fix-windows-agent-cpu-breaches/test/scenario/multiline/dotnet-win.yaml
kubectl apply -f https://raw.githubusercontent.com/microsoft/Docker-Provider/gangams/fix-windows-agent-cpu-breaches/test/scenario/multiline/golang-win.yaml
kubectl apply -f https://raw.githubusercontent.com/microsoft/Docker-Provider/gangams/fix-windows-agent-cpu-breaches/test/scenario/multiline/java-win.yaml
kubectl apply -f https://raw.githubusercontent.com/microsoft/Docker-Provider/gangams/fix-windows-agent-cpu-breaches/test/scenario/multiline/python-win.yaml

**Expected behavior**
CPU shouldnt grow increasingly grow and usage should be same as fluent-bit version 2.0.14.

**Screenshots**
CPU Usage  with fluent-bit version 2.0.14 vs fluent-bit version 2.2.2
![image](https://github.com/fluent/fluent-bit/assets/20712167/1507c5b8-7e56-4d44-b4ba-112b959c3b27)

**Your Environment**
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used: Fluent-bit 2.0.14 and Fluent-bit 2.2.2
* Configuration: Kubernetes with Windows Server 2019
* Environment name and version (e.g. Kubernetes? What version?): Kubernetes, version : v1.26.12
* Server type and version: Windows Server 2019
* Operating System and version:
* Filters and plugins:  Tail, Multiline and Output

**Additional context**
<!--- How has this issue affected you? What are you trying to accomplish? -->
<!--- Providing context helps us come up with a solution that is most useful in the real world -->

[tieptoi]

Same as my issue #8925

[yangliuqing5470]

because multiline parser is slower with fluent-bit version >=2.1.0，and i compare the commit，it's "filter_multiline: updated to use the new log event abstraction layer" commit casue the issue