search

prometheus / jmx_exporter

A process for exposing JMX Beans via HTTP for Prometheus consumption
Apache License 2.0
3.02k stars 1.2k forks source link

Issue collecting attributes with text values from normal attributes and composite data #606

Closed IanWilkins closed 3 years ago

IanWilkins commented 3 years ago

I'm trying to collect metrics for multiple beans and am not able to collect the values for attributes that have textual values. Specifically any attribute with name=status is not being recorded, there are other examples as well, but this is symptomatic of the likely issue.

If I examine the bean in jconsole (see image) I can see the complete set of attribute values for the mbean com.ibm.team.foundation.diagnostic:name=rm,type=diagnosticMetrics,testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics

image

If I look at the target metrics (http://...:9081/metrics), I only seeing the following attributes being collected.

com_ibm_team_foundation_diagnostic_startTime{contextRoot="rm",testId="com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics",type="diagnosticMetrics",} 1.62220933089E9
com_ibm_team_foundation_diagnostic_duration{contextRoot="rm",testId="com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics",type="diagnosticMetrics",} 0.0
com_ibm_team_foundation_diagnostic_mbeanCreationTimestamp{contextRoot="rm",testId="com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics",type="diagnosticMetrics",} 1.622224266612E9
com_ibm_team_foundation_diagnostic_port{contextRoot="rm",testId="com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics",type="diagnosticMetrics",} -1.0

Other Files app4-jmx-metrics.txt jmx-exporter-config.yml.txt

fstab commented 3 years ago

Please click on the status attribute in jconsole and let me know the Type of the status attribute.

IanWilkins commented 3 years ago

The status attribute is of type java.lang.String

image

I've done some more digging/testing. If I run the scrapper in standalone mode as described in the README.md, I'm seeing all the attributes being scraped.

com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]statusDesc: ignored
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]contextRoot: rm
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]detailStatusDesc: 
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]duration: 0
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]mbeanCreationTimestamp: 1.622551867084E9
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]port: -1
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]domain: com.ibm.team.foundation.diagnostic
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]host: elmtest.rtp.raleigh.ibm.com
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]startTime: 1.622547732975E9
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]testId: com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]nodeId: rm
com.ibm.team.foundation.diagnostic{name=rm, type=diagnosticMetrics, testId=com.ibm.team.jfs.indexing.service.internal.diagnostics.IndicesDiagnostics}[]status: OK

I've tried multiple changes to the configuration yml file. But I'm just not seeing all the attributes for this bean being displayed. (This is happening with other beans as well, but this is a good example of the problem)

fstab commented 3 years ago

Prometheus metrics always have a number as their value. You can have textual information in labels, but the value must be a number. That's why using the status field of type String as a metric value does not work.

fstab commented 3 years ago

Good news: There is actually a way to represent states as boolean numbers, it's described here https://www.robustperception.io/converting-string-states-to-booleans-with-the-jmx-exporter

IanWilkins commented 3 years ago

Thanks for the info. I'll close this issue as Prometheus and the exporter are working as designed.