influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.89k stars 5.6k forks source link

feat(processors.converter): added support for base64 encoded ieee float32 types #16214

Open vkrasnici opened 5 days ago

vkrasnici commented 5 days ago

Summary

When using Cisco Telemetry MDT (cisco_telemetry_mdt) Input Plugin in combination with OpenConfig power Yang model (openconfig-platform:components/component/power-supply) the data is received but the power values are Base64 Encoded IEEE 754 Float32. The format received looks like below:

{"fields":{"data_json_content_state_openconfig-platform-psu:input-current":"QWAAAA==","data_json_timestamp":"1731938073759"},"name":"file","tags":{"host":"YJ4"},"timestamp":1732125250}
{"fields":{"data_json_content_state_openconfig-platform-psu:output-current":"QcAAAA==","data_json_timestamp":"1731938073763"},"name":"file","tags":{"host":"YJ4"},"timestamp":1732125250}
{"fields":{"data_json_content_state_openconfig-platform-psu:input-current":"QVAAAA==","data_json_timestamp":"1731938073766"},"name":"file","tags":{"host":"YJ4"},"timestamp":1732125250}
{"fields":{"data_json_content_state_openconfig-platform-psu:output-power":"RKeAAA==","data_json_timestamp":"1731938073773"},"name":"file","tags":{"host":"YJ4"},"timestamp":1732125250}

This makes the data unusable for metrics, as metrics should be represented by float/int (number). Using parser.json and parse_fields_base64 converts b64 to byte/str, but that's not enough to convert it to metrics, hence the motivation to add the converter support.

Checklist

Related issues

Closes #16213

telegraf-tiger[bot] commented 19 hours ago

Download PR build artifacts for linux_amd64.tar.gz, darwin_arm64.tar.gz, and windows_amd64.zip. Downloads for additional architectures and packages are available below.

:relaxed: This pull request doesn't significantly change the Telegraf binary size (less than 1%)

:package: Click here to get additional PR build artifacts ## Artifact URLs | DEB | RPM | TAR GZ | ZIP | |------------------------------------------------------------------------------------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | [amd64.deb](https://output.circle-artifacts.com/output/job/6efe1dcd-feb6-4a2a-9f97-66bd1df34b3d/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_amd64.deb) | [aarch64.rpm](https://output.circle-artifacts.com/output/job/2ddb93bc-f5f8-4eb8-89b1-58645d8d0bda/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.aarch64.rpm) | [darwin_amd64.tar.gz](https://output.circle-artifacts.com/output/job/e3c0fc47-7a21-47bb-aa53-b663b840dac6/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_darwin_amd64.tar.gz) | [windows_amd64.zip](https://output.circle-artifacts.com/output/job/5e4f753d-f285-4858-a1e1-a953255d8f97/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_windows_amd64.zip) | | [arm64.deb](https://output.circle-artifacts.com/output/job/2ddb93bc-f5f8-4eb8-89b1-58645d8d0bda/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_arm64.deb) | [armel.rpm](https://output.circle-artifacts.com/output/job/fe4586a9-4044-41a9-89b8-4517381da98f/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.armel.rpm) | [darwin_arm64.tar.gz](https://output.circle-artifacts.com/output/job/0f043cf2-4037-4f8e-a767-b94fca3583e1/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_darwin_arm64.tar.gz) | [windows_arm64.zip](https://output.circle-artifacts.com/output/job/5e4f753d-f285-4858-a1e1-a953255d8f97/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_windows_arm64.zip) | | [armel.deb](https://output.circle-artifacts.com/output/job/fe4586a9-4044-41a9-89b8-4517381da98f/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_armel.deb) | [armv6hl.rpm](https://output.circle-artifacts.com/output/job/da0240aa-d578-45a8-826d-f01671ab184a/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.armv6hl.rpm) | [freebsd_amd64.tar.gz](https://output.circle-artifacts.com/output/job/6efe1dcd-feb6-4a2a-9f97-66bd1df34b3d/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_freebsd_amd64.tar.gz) | [windows_i386.zip](https://output.circle-artifacts.com/output/job/5e4f753d-f285-4858-a1e1-a953255d8f97/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_windows_i386.zip) | | [armhf.deb](https://output.circle-artifacts.com/output/job/da0240aa-d578-45a8-826d-f01671ab184a/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_armhf.deb) | [i386.rpm](https://output.circle-artifacts.com/output/job/6fbeb71f-bfeb-4b73-bcff-63aae61103eb/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.i386.rpm) | [freebsd_armv7.tar.gz](https://output.circle-artifacts.com/output/job/da0240aa-d578-45a8-826d-f01671ab184a/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_freebsd_armv7.tar.gz) | | | [i386.deb](https://output.circle-artifacts.com/output/job/6fbeb71f-bfeb-4b73-bcff-63aae61103eb/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_i386.deb) | [ppc64le.rpm](https://output.circle-artifacts.com/output/job/9bf4e904-e183-45b4-be96-b960c2fccc7f/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.ppc64le.rpm) | [freebsd_i386.tar.gz](https://output.circle-artifacts.com/output/job/6fbeb71f-bfeb-4b73-bcff-63aae61103eb/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_freebsd_i386.tar.gz) | | | [mips.deb](https://output.circle-artifacts.com/output/job/1bb109d3-5a88-484b-b099-70126db270d6/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_mips.deb) | [riscv64.rpm](https://output.circle-artifacts.com/output/job/86144b2d-53db-4997-a223-4520f8bbd312/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.riscv64.rpm) | [linux_amd64.tar.gz](https://output.circle-artifacts.com/output/job/6efe1dcd-feb6-4a2a-9f97-66bd1df34b3d/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_amd64.tar.gz) | | | [mipsel.deb](https://output.circle-artifacts.com/output/job/b0884847-5463-4f2a-ba2b-3fc046c9f14b/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_mipsel.deb) | [s390x.rpm](https://output.circle-artifacts.com/output/job/4a6075de-740f-444a-8d18-312c0adc3f50/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.s390x.rpm) | [linux_arm64.tar.gz](https://output.circle-artifacts.com/output/job/2ddb93bc-f5f8-4eb8-89b1-58645d8d0bda/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_arm64.tar.gz) | | | [ppc64el.deb](https://output.circle-artifacts.com/output/job/9bf4e904-e183-45b4-be96-b960c2fccc7f/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_ppc64el.deb) | [x86_64.rpm](https://output.circle-artifacts.com/output/job/6efe1dcd-feb6-4a2a-9f97-66bd1df34b3d/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160-0.x86_64.rpm) | [linux_armel.tar.gz](https://output.circle-artifacts.com/output/job/fe4586a9-4044-41a9-89b8-4517381da98f/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_armel.tar.gz) | | | [riscv64.deb](https://output.circle-artifacts.com/output/job/86144b2d-53db-4997-a223-4520f8bbd312/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_riscv64.deb) | | [linux_armhf.tar.gz](https://output.circle-artifacts.com/output/job/da0240aa-d578-45a8-826d-f01671ab184a/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_armhf.tar.gz) | | | [s390x.deb](https://output.circle-artifacts.com/output/job/4a6075de-740f-444a-8d18-312c0adc3f50/artifacts/0/build/dist/telegraf_1.33.0~9cc2d160-0_s390x.deb) | | [linux_i386.tar.gz](https://output.circle-artifacts.com/output/job/6fbeb71f-bfeb-4b73-bcff-63aae61103eb/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_i386.tar.gz) | | | | | [linux_mips.tar.gz](https://output.circle-artifacts.com/output/job/1bb109d3-5a88-484b-b099-70126db270d6/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_mips.tar.gz) | | | | | [linux_mipsel.tar.gz](https://output.circle-artifacts.com/output/job/b0884847-5463-4f2a-ba2b-3fc046c9f14b/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_mipsel.tar.gz) | | | | | [linux_ppc64le.tar.gz](https://output.circle-artifacts.com/output/job/9bf4e904-e183-45b4-be96-b960c2fccc7f/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_ppc64le.tar.gz) | | | | | [linux_riscv64.tar.gz](https://output.circle-artifacts.com/output/job/86144b2d-53db-4997-a223-4520f8bbd312/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_riscv64.tar.gz) | | | | | [linux_s390x.tar.gz](https://output.circle-artifacts.com/output/job/4a6075de-740f-444a-8d18-312c0adc3f50/artifacts/0/build/dist/telegraf-1.33.0~9cc2d160_linux_s390x.tar.gz) | |