search

edgexfoundry / app-functions-sdk-go

Owner: Applications WG
Apache License 2.0
43 stars 83 forks source link

Got "duplicate metric" error after updating writable.telemetry.metrics.MqttExportSize #1537

Closed cherrycl closed 9 months ago

cherrycl commented 10 months ago

🐞 Bug Report

Affected Services [REQUIRED]

The issue is located in: app-service ### Is this a regression? Yes, the previous version in which this bug was not present was: Napa ### Description and Minimal Reproduction [**REQUIRED**] 1. Download the [compose file](https://raw.githubusercontent.com/edgexfoundry/edgex-compose/main/docker-compose-no-secty-with-app-sample.yml) and attached docker-compose-mqtt-broker.yml file. [docker-compose-mqtt-broker.yml.zip](https://github.com/edgexfoundry/app-functions-sdk-go/files/13896513/docker-compose-mqtt-broker.yml.zip) 2. Deploy EdgeX with the compose files `docker compose -f docker-compose-no-secty-with-app-sample.yml -f docker-compose-mqtt-broker.yml up -d`. 3. Update `app-sample/Writable/Pipeline/Functions/MQTTExport/Parameters/BrokerAddress` to `tcp://edgex-mqtt-broker:1883` by Consul GUI 4. Update `app-sample/Writable/Pipeline/ExectionOrder` to `MqttExport` by Consul GUI 5. Create new key `Writable/Telemetry/Metrics/MqttExportSize` and set value to `true` under app-sample 6. Subscribe `edgex.telemetry.app-sample.MqttExportSize` by redis-cli and wait 2 minutes to receive the MqttExportSize metric ## 🔥 Exception or Error The metric value of histogram-count are always the same, but events keep exporting. 

Found the following logs after step 4:
level=INFO ts=2024-01-11T02:15:02.325266444Z app=app-sample source=helpers.go:47 msg="MqttExportErrors-tcp://edgex-mqtt-broker:1883/mytopic metric has been registered and will be reported (if enabled)"
level=INFO ts=2024-01-11T02:15:02.325349182Z app=app-sample source=helpers.go:47 msg="MqttExportSize-tcp://edgex-mqtt-broker:1883/mytopic metric has been registered and will be reported (if enabled)"
And logs after step 5:
level=ERROR ts=2024-01-11T02:17:12.31746129Z app=app-sample source=helpers.go:43 msg="Unable to register metric MqttExportErrors-tcp://edgex-mqtt-broker:1883/mytopic. Collection will continue, but metric will not be reported: duplicate metric: MqttExportErrors-tcp://edgex-mqtt-broker:1883/mytopic"
level=ERROR ts=2024-01-11T02:17:12.317548129Z app=app-sample source=helpers.go:43 msg="Unable to register metric MqttExportSize-tcp://edgex-mqtt-broker:1883/mytopic. Collection will continue, but metric will not be reported: duplicate metric: MqttExportSize-tcp://edgex-mqtt-broker:1883/mytopic"
## 🌍 Your Environment **Deployment Environment:** Ubuntu / x86 **EdgeX Version [**REQUIRED**]:** Main **Anything else relevant?** `HttpExportSize`, `HttpExportErrors`, `MqttExportErrors` have the same issue.
lenny-goodell commented 9 months ago

@cherrycl , I was not able to reproduce the Registration errors since it is a timing issue, but do see how this could occur. I have an idea on how to detect and ignore this issue without the need for locking, which will cause performance issue.

cherrycl commented 9 months ago

Error duplicate metric was gone, but the The value of histogram-count for MqttExportSize is still incorrect. Create new issue #1540 for it.