search

edgexfoundry / app-functions-sdk-go

Owner: Applications WG
Apache License 2.0
43 stars 83 forks source link

MqttExportSize metric stop to collect data after set the writable.telemetry.metrics.MqttExportSize=true #1540

Closed cherrycl closed 9 months ago

cherrycl commented 9 months ago

🐞 Bug Report

Affected Services [REQUIRED]

The issue is located in: app-service ### Is this a regression? Yes, the previous version in which this bug was not present was: Napa ### Description and Minimal Reproduction [**REQUIRED**] Reproduce steps are the same as #1537, but correct the `ExectionOrder` value to `MQTTExport` on step4. 1. Download the [compose file](https://raw.githubusercontent.com/edgexfoundry/edgex-compose/main/docker-compose-no-secty-with-app-sample.yml) and attached docker-compose-mqtt-broker.yml file. [docker-compose-mqtt-broker.yml.zip](https://github.com/edgexfoundry/app-functions-sdk-go/files/13896513/docker-compose-mqtt-broker.yml.zip) 2. Deploy EdgeX with the compose files `docker compose -f docker-compose-no-secty-with-app-sample.yml -f docker-compose-mqtt-broker.yml up -d`. 3. Update `app-sample/Writable/Pipeline/Functions/MQTTExport/Parameters/BrokerAddress` to `tcp://edgex-mqtt-broker:1883` by Consul GUI 4. Update `app-sample/Writable/Pipeline/ExectionOrder` to `MQTTExport` by Consul GUI 5. Create new key `Writable/Telemetry/Metrics/MqttExportSize` and set value to `true` under app-sample 6. Subscribe `edgex.telemetry.app-sample.MqttExportSize` by redis-cli and wait 2 minutes to receive the MqttExportSize metric ## 🔥 Exception or Error MqttExportSize metric start to collect data after setting ExectionOrder to MQTTExport, but stopped after the set the writable.telemetry.metrics.MqttExportSize=true The value of histogram-count for MqttExportSize always are the the same, even MQTTExport keeps exporting event. ## 🌍 Your Environment **Deployment Environment:**Ubuntu / x86 **EdgeX Version [**REQUIRED**]:**Main **Anything else relevant?** `HttpExportSize` has the same issue.
lenny-goodell commented 9 months ago

@cherrycl , can you try swapping steps 4 & 5? The dynamic rebuilding of the function pipeline is getting trigger by creating the new key (which it shouldn't) and may be causing issue with the metric.

cherrycl commented 9 months ago

@lenny-intel Thanks. Yes, the metric works correctly after swapping step4 and step5.

lenny-goodell commented 9 months ago

@lenny-intel Thanks. Yes, the metric works correctly after swapping step4 and step5.

Great. We'll leave this issue open to address the issue of rebuilding the the pipelines when something unrelated changed in Writable.

cherrycl commented 9 months ago

@lenny-intel I retested this issue on latest app-service image, the issue is still existed when adding Writable/Telemetry/Metrics/MqttExportSize metrics to true from app-sample, but not when changing the value of app-services/Writable/Telemetry/Metrics/MqttExportSize to true from core-common-config-bootstrapper.

cherrycl commented 9 months ago

Logs after setting MqttExportSize for reference.

level=INFO ts=2024-01-29T03:06:49.262318743Z app=app-sample source=config.go:969 msg="Writable configuration has been updated from the Configuration Provider"
level=INFO ts=2024-01-29T03:06:49.262386217Z app=app-sample source=configupdates.go:63 msg="Processing App Service configuration updates"
level=INFO ts=2024-01-29T03:06:49.262462735Z app=app-sample source=mqttsecret.go:129 msg="Initializing MQTT Client"
level=INFO ts=2024-01-29T03:06:49.262484782Z app=app-sample source=mqttsecret.go:318 msg="Attempting to Pre-Connect to mqtt server for export"
level=INFO ts=2024-01-29T03:06:49.266781672Z app=app-sample source=mqttsecret.go:335 msg="Pre-Connected to mqtt server for export"
level=INFO ts=2024-01-29T03:06:49.268083027Z app=app-sample source=runtime.go:121 msg="Transforms set for `default-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.268192532Z app=app-sample source=runtime.go:135 msg="Topics '[#]' set for `default-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.268232496Z app=app-sample source=runtime.go:121 msg="Transforms set for `float-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.26833855Z app=app-sample source=runtime.go:135 msg="Topics '[events/device/device-virtual/Random-Float-Device/#]' set for `float-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.268386249Z app=app-sample source=runtime.go:121 msg="Transforms set for `int8-16-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.268455193Z app=app-sample source=runtime.go:135 msg="Topics '[events/device/device-virtual/+/+/Int8 events/device/device-virtual/+/+/Int16]' set for `int8-16-pipeline` pipeline"
level=INFO ts=2024-01-29T03:06:49.268562528Z app=app-sample source=configupdates.go:153 msg="Configurable Pipeline successfully reloaded from new configuration"
lenny-goodell commented 9 months ago

I root caused the issue and reopend https://github.com/edgexfoundry/app-functions-sdk-go/issues/1541