Open zekai-rai opened 3 weeks ago
Pinging code owners:
receiver/otlpjsonfile: @djaglowski @atoulme
See Adding Labels via Comments if you do not have permissions to add labels yourself.
/label receiver/filelog help-wanted -receiver/otlpjsonfile
Pinging code owners for receiver/filelog: @djaglowski. See Adding Labels via Comments if you do not have permissions to add labels yourself.
Component(s)
exporter/file, receiver/otlpjsonfile
What happened?
Description
We have a special multiple-part system that requires us to export telemetry into the file system and get picked up again with another pipeline later. We are using the File exporter to export logs in one pipeline, and another pipeline is using filelog receiver to read it.
(This description removes some irrelevant setup and might not truly reflect what we are doing.)
We noticed that with rotation turned on in file exporter, some hidden null characters (\u0000) are added to the beginning of the file. We are unsure if it's an issue in the receiver or the exporter.
On a related note, we observed a similar behavior in another pipeline where logs are rotated by LogRotate out of the Otel collector pipeline. In this case, we understand that the issue was caused by the fact that file are being actively written to by our process while logrotate is truncating it. However, we do not understand why it's an issue with filelog receiver and file exporter setup.
Steps to Reproduce
Pipeline 1: configuring file exporter to export with rotation. Pipeline 2 run elsewhere: configuring file log received to read from the rotated files. Check the content of pipeline 2.
Expected Result
Normal logs as being originally produced.
Actual Result
\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000\u0000{"content":"my valid json. just for demo, not real structure."} {"content":"my valid json. just for demo, not real structure."} {"content":"my valid json. just for demo, not real structure."} {"content":"my valid json. just for demo, not real structure."}
Collector version
0.98.0
Environment information
Environment
OS: (e.g., "Ubuntu 20.04") Compiler(if manually compiled): (e.g., "go 14.2")
OpenTelemetry Collector configuration
Log output
No response
Additional context
No response