logsCollection storage extension

[povilasv]

What do you think if we configure storage extension for the filelog receiver using logsCollection preset?

Then we would set up the file_storage extension for it?

As I understand it stores the last file offsets to a file.

Something like:

extensions:
  file_storage:
    directory: /var/log/pods/
receivers:
  filelog:
    include: [ /var/log/pods/*/*/*.log ]
    ...
    storage: file_storages

Though there is one question - where do we store offsets db?

[TylerHelmuth]

I think this is a good idea and safe since the file_storage extension is beta. I would opt to but this feature behind a setting in the logCollection preset so that users have to opt in to this extra component.

[povilasv]

I keep trying to make this extension work, but for some reason I get panics without any errors:

https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/16469 https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/16471