network-analytics / mdt-dialout-collector

Model-Driven Telemetry - Collecting <multi-vendor> metrics via gRPC dialout
MIT License
27 stars 8 forks source link

Issue with null values in Juniper jRPC message #36

Open sgaragan opened 2 months ago

sgaragan commented 2 months ago

For some reason, the devices we are using sometimes sends null characters in the gRPC message but we cannot seem to print out what that message is to understand what is being sent. The error shows up in the pmtelemetryd logs as shown below. Not sure if we can enable the config parameter (JSON_ALLOW_NUL) to let it through so we can see it that way but if that is possible, could someone let us know how to set it?

Thanks, Sean

[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] Srv::JuniperStream::Start() - Thread-ID: 139969992492800
DEBUG ( pmtelemetryd-grpc/core ): [/etc/pmacct/librdkafka.conf] Reading librdkafka topic configuration.
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: request.required.acks = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: request.timeout.ms = 30000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: message.timeout.ms = 300000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: queuing.strategy = fifo
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: produce.offset.report = false
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: partitioner = consistent_random
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: compression.codec = inherit
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: compression.level = -1
[2024-07-24 13:16:52.465] [multi-logger] [info] [JuniperStream::Start()] x.x.x.x JuniperExtension, parsing successful
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.commit.enable = true
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.commit.interval.ms = 60000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.offset.reset = largest
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.path = .
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.sync.interval.ms = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.method = broker
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.path = .
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.sync.interval.ms = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.method = broker
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: consume.callback.max.messages = 0
DEBUG ( pmtelemetryd-grpc/core ): JSON error: \u0000 is not allowed without JSON_ALLOW_NUL near '"\u0000"' (1/230/230: <string>)[2024-07-24 13:16:52.465] [multi-logger] [info] [MetaData] data-manipulation: x.x.x.x meta-data added successfully
[2024-07-24 13:16:52.465] [multi-logger] [info] [ZmqPusher] data-delivery: message successfully sent
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] Srv::JuniperStream::Start() - Thread-ID: 139969992492800
[2024-07-24 13:16:52.465] [multi-logger] [info] [JuniperStream::Start()] x.x.x.x JuniperExtension, parsing successful
[2024-07-24 13:16:52.465] [multi-logger] [info] [MetaData] data-manipulation: x.x.x.x meta-data added successfully
[2024-07-24 13:16:52.465] [multi-logger] [info] [ZmqPusher] data-delivery: message successfully sent
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] Srv::JuniperStream::Start() - Thread-ID: 139969992492800
[2024-07-24 13:16:52.465] [multi-logger] [info] [JuniperStream::Start()] x.x.x.x JuniperExtension, parsing successful
[2024-07-24 13:16:52.465] [multi-logger] [info] [MetaData] data-manipulation: x.x.x.x meta-data added successfully
[2024-07-24 13:16:52.465] [multi-logger] [info] [ZmqPusher] data-delivery: message successfully sent
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] Srv::JuniperStream::Start() - Thread-ID: 139969992492800
DEBUG ( pmtelemetryd-grpc/core ): [/etc/pmacct/librdkafka.conf] Reading librdkafka topic configuration.
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: request.required.acks = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: request.timeout.ms = 30000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: message.timeout.ms = 300000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: queuing.strategy = fifo
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: produce.offset.report = false
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: partitioner = consistent_random
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: compression.codec = inherit
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: compression.level = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.commit.enable = true
[2024-07-24 13:16:52.465] [multi-logger] [info] [JuniperStream::Start()] x.x.x.x JuniperExtension, parsing successful
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.commit.interval.ms = 60000
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: auto.offset.reset = largest
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.path = .
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.sync.interval.ms = -1
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: offset.store.method = broker
DEBUG ( pmtelemetryd-grpc/core ): librdkafka 'ietf.grpc-raw' topic config: consume.callback.max.messages = 0
[2024-07-24 13:16:52.465] [multi-logger] [info] [MetaData] data-manipulation: x.x.x.x meta-data added successfully
DEBUG ( pmtelemetryd-grpc/core ): JSON error: \u0000 is not allowed without JSON_ALLOW_NUL near '"\u0000"' (1/232/232: <string>)[2024-07-24 13:16:52.465] [multi-logger] [info] [ZmqPusher] data-delivery: message successfully sent
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] constructor: JuniperStream()
[2024-07-24 13:16:52.465] [multi-logger] [debug] Srv::JuniperStream::Start() - Thread-ID: 139969992492800
[2024-07-24 13:16:52.465] [multi-logger] [info] [JuniperStream::Start()] x.x.x.x JuniperExtension, parsing successful
[2024-07-24 13:16:52.465] [multi-logger] [info] [MetaData] data-manipulation: x.x.x.x meta-data added successfully
[2024-07-24 13:16:52.465] [multi-logger] [info] [ZmqPusher] data-delivery: message successfully sent
[2024-07-24 13:16:52.465] [multi-logger] [debug] destructor: ~JuniperStream()