[version=3.1.4][kubernetes_events] after an 410 error with the api it stops working

[rurus9]

Bug Report

With this config:

    [Input]
        name kubernetes_events
        Alias input:kubernetes_events
        # add the tag "log.k8s_events" to all events coming from this input
        tag log.k8s_events
        # Set a database file to keep track of recorded Kubernetes events
        DB /storage/log.kubernetes_events.db
        DB.Sync Normal
        # ask k8s API for updates every 10 seconds
        interval_sec 10
        ## fetch at most 250 items per requests (pagination)
        kube_request_limit 250
        # API Server end-point
        kube_url https://kubernetes.default.svc
        # storage on disk
        storage.type filesystem
        # Specifies if the input plugin should be paused (stop ingesting new data) when the storage.max_chunks_up value is reached.
        storage.pause_on_chunks_overlimit On

fluent-bit stop reading new events after error http_status=410 from API (and it is quite a hard-working K8s cluster):

[2024/08/30 09:54:07] [error] [input:kubernetes_events:input:kubernetes_events] http_status=410:
{"kind":"Status","apiVersion":"v1","metadata":{"continue":"REDACTED"},"status":"Failure","message":"The provided continue parameter is too old to display a consistent list result. You can start a new list without the continue parameter, or use the continue token in this response to retrieve the remainder of the results. Continuing with the provided token results in an inconsistent list - objects that were created, modified, or deleted between the time the first chunk was returned and now may show up in the list.","reason":"Expired","code":410}

Expected behavior fluent-bit should deal with this error

[edsiper]

cc: @ryanohnemus

[ryanohnemus]

This was fixed in 3.1.5

[rurus9]

Indeed, in version 3.1.5 it works properly, after the error, it resumed working after some time.

[2024/09/09 11:09:28] [error] [input:kubernetes_events:input:kubernetes_events] http_status=410:
{"kind":"Status","apiVersion":"v1","metadata":{"continue":REDACTED"},"status":"Failure","message":"The provided continue parameter is too old to display a consistent list result. You can start a new list without the continue parameter, or use the continue token in this response to retrieve the remainder of the results. Continuing with the provided token results in an inconsistent list - objects that were created, modified, or deleted between the time the first chunk was returned and now may show up in the list.","reason":"Expired","code":410}

[2024/09/09 11:17:53] [ info] [input:kubernetes_events:input:kubernetes_events] Requesting /api/v1/events?watch=1&resourceVersion=917334540

Thank you!