Azure / iotedge

The IoT Edge OSS project
MIT License
1.46k stars 459 forks source link

Azure IOT hub edge agent metrics not sending metrics related to memory usage by the module when using raspberry Pi. #7159

Closed thechromosomes closed 9 months ago

thechromosomes commented 11 months ago

Expected Behavior

Azure IOT hub edge agent metrics should send metrics related to memory usage by the module

Current Behavior

Azure IOT hub edge agent metrics not sending metrics related to memory usage by the module

Steps to Reproduce

Provide a detailed set of steps to reproduce the bug.

  1. deploy edge runtime on Raspberry Pi Bullseye
  2. expose the edge agent metrics
  3. check _edgeAgent_used_memorybytes
Click here ``` ![image](https://github.com/Azure/iotedge/assets/48309910/23661341-3bcf-470a-8bc2-c31d90ff1ccc) ```

Device Information

Runtime Versions

Additional Information

REF: https://learn.microsoft.com/en-us/answers/questions/1424158/azure-iot-hub-edge-agent-metrics-not-sending-metri

nyanzebra commented 11 months ago

Hi @thechromosomes, few questions.

  1. Do you have any allow/disallow rules that might be preventing the metric from being collected? See here for Allowed/Blocked metrics
  2. If you scrape http://edgeAgent:9600/metrics do you see the metric? If not, what metrics do you see?
  3. If you are using the MetricCollector, would you mind sharing what your config for the module looks like?
thechromosomes commented 11 months ago

Hi @thechromosomes, few questions.

  1. Do you have any allow/disallow rules that might be preventing the metric from being collected? See here for Allowed/Blocked metrics
  2. If you scrape http://edgeAgent:9600/metrics do you see the metric? If not, what metrics do you see?
  3. If you are using the MetricCollector, would you mind sharing what your config for the module looks like?

Hi @nyanzebra,

I am using custom metrics solution using Prometheus. exact same configuration is working for non raspberry Pi devices(like ubuntu), raspberry Pi is also producing the memory metrics but its only producing the metrics related to the host not for all the module deployed in the edge runtime.

Raspberry pi metrics (memory usage) image

ubuntu metrics (memory usage) image

nyanzebra commented 11 months ago

@thechromosomes, so just to understand fully, if you run iotedge list from both devices you have same modules running and from grafana you are using edgeAgent_used_memory_bytes for the metric? If so, would you mind providing debug logs for edgeAgent on the pi device?

jlian commented 10 months ago

@thechromosomes any updates

jlian commented 9 months ago

Closing for now. We can reopen when we get the logs