Closed Nirwx closed 1 year ago
Hi,
Looks to be caused during the xpath parser's Init(), when we pass the files to the github.com/jhump/protoreflect/desc/protoparse
library to parse the files. Would you be willing to share both sci-message.proto
and /opt/samples/message-3.proto.dump
? If you don't want to do it pulibcally, you can send me mail at jpowers at influxdata.com
Hi, Thanks. I've sent an email.
Thank you! Will take a look shortly.
Alright, with the files you sent me I built the docker image as follows:
FROM telegraf:1.26
COPY telegraf.conf /etc/telegraf/telegraf.conf
COPY proto/ /opt/proto
COPY message-3.proto.dump /opt/samples/
WORKDIR /opt/proto
The two differences being the two last COPY commands. The first one is a directory containing the protofiles you sent me. The second is the direct dump file you sent me as well.
And when I run:
❯ docker run -it telegraf:test telegraf
2023-09-29T16:11:34Z I! Loading config: /etc/telegraf/telegraf.conf
2023-09-29T16:11:34Z I! Starting Telegraf 1.26.3
2023-09-29T16:11:34Z I! Available plugins: 235 inputs, 9 aggregators, 27 processors, 22 parsers, 57 outputs, 2 secret-stores
2023-09-29T16:11:34Z I! Loaded inputs: file
2023-09-29T16:11:34Z I! Loaded aggregators:
2023-09-29T16:11:34Z I! Loaded processors:
2023-09-29T16:11:34Z I! Loaded secretstores:
2023-09-29T16:11:34Z I! Loaded outputs: file
2023-09-29T16:11:34Z I! Tags enabled:
2023-09-29T16:11:34Z I! [agent] Config: Interval:5s, Quiet:false, Hostname:"", Flush Interval:5s
2023-09-29T16:11:34Z D! [agent] Initializing plugins
2023-09-29T16:11:34Z D! [agent] Connecting outputs
2023-09-29T16:11:34Z D! [agent] Attempting connection to [outputs.file]
2023-09-29T16:11:34Z D! [agent] Successfully connected to outputs.file
2023-09-29T16:11:34Z D! [agent] Starting service inputs
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] Could not find "sci.SciMessage"... Known messages:
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] ScgSessMgrPubIpc
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] com.ruckuswireless.scg.protobuf.icx
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] com.ruckuswireless.scg.protobuf.icx
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] com.ruckuswireless.scg.protobuf.storage
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] com.ruckuswireless.scg.protobuf.storage
2023-09-29T16:11:35Z I! [parsers.xpath_protobuf::file] google.protobuf
2023-09-29T16:11:35Z E! [inputs.file] Error in plugin: could not instantiate parser: proto: not found
^C2023-09-29T16:11:38Z D! [agent] Stopping service inputs
2023-09-29T16:11:38Z D! [agent] Input channel closed
2023-09-29T16:11:38Z I! [agent] Hang on, flushing any cached metrics before shutdown
2023-09-29T16:11:38Z D! [outputs.file] Buffer fullness: 0 / 30000 metrics
2023-09-29T16:11:38Z I! [agent] Stopping running outputs
2023-09-29T16:11:38Z D! [agent] Stopped Successfully
That is because I need to change xpath_protobuf_type = "sci.SciMessage"
to xpath_protobuf_type = "SciMessage"
Is it possible to remove Docker builds from the equation and have you run it locally as well? Ideally, with v1.28.1 with the following config:
[agent]
debug = true
[[inputs.file]]
files = ["samples/mqtt_log/dumps/message-3.proto.dump"]
data_format = "xpath_protobuf"
xpath_protobuf_import_paths = [".", "proto/base"]
xpath_protobuf_file = "sci-message.proto"
xpath_protobuf_type = "SciMessage"
xpath_print_document = true
[[outputs.file]]
With that I was able to get the XML document to print and you would be ready to start creating your config to parse and generate metrics.
Looking at the upstream library, the trace looks very much like https://github.com/jhump/protoreflect/issues/572
Hi, I still run into the same issue with the docker container. I've also tried locally v1.28.1 using the telegraf conf you shared, and same error.
Note that I'm running this on a Mac M1 (Sonoma Version 14.0).
@powersj can you please try to update protoreflect
to https://github.com/jhump/protoreflect/pull/574 and see if the issue goes away?
Need https://github.com/jhump/protoreflect/issues/572 to be fixed...
@Nirwx just to be really sure: Did you set xpath_protobuf_type = "SciMessage"
as @powersj suggested? That is the correct message name...
@Nirwx please check if PR #14085 fixes your issue as it contains a fixed version of github.com/jhump/protoreflect
. A binary should be available in the PR once CI finished all tests successfully.
Relevant telegraf.conf
Logs from Telegraf
System info
telegraf container version 1.26 to 1.28.1
Docker
FROM telegraf:1.26 COPY telegraf.conf /etc/telegraf/telegraf.conf COPY proto/base /opt/proto COPY samples/mqtt_log/dumps /opt/samples WORKDIR /opt/proto
Steps to reproduce
Expected behavior
Protobuf message written to ["/tmp/output.txt"]
Actual behavior
"Invalid Memory Address"
2023-09-29T15:08:04Z D! [agent] Starting service inputs panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0x3cffe24]
Additional info
No response