MBeanInfo parsing errors: wrong metrics type detection for C* 2.2.8

[kyxap1]

Hello,

there are numerous problems with MBeanInfo parser for Cassandra 2.2.8. Both exporter and cassandra runtime logs are attached.

Could you confirm this behaviour with 2.x branch? Thanks!

[main] DEBUG com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot parse java.lang:type=MemoryPool,name=Compressed Class Space as it as an unknown type java.lang.String with value NON_HEAP
[main] DEBUG com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot parse java.lang:type=MemoryPool,name=Compressed Class Space as it as an unknown type javax.management.ObjectName with value java.lang:type=MemoryPool,name=Compressed Class Space
[main] DEBUG com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot parse org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=HitRate as it as an unknown type java.lang.Object with value NaN
[main] DEBUG com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot parse java.lang:type=MemoryManager,name=Metaspace Manager as it as an unknown type [Ljava.lang.String; with value [Metaspace, Compressed Class Space]

cassandra.system.log

cassandra_exporter.debug.log

[erebe]

Hello,

Yes this is normal. There is a lot of Mbeans that cannot be converted into metrics, so values that contains something that cannot be converted into a numeric is discarded (string, complex objects, ...)

[kyxap1]

Can it be the reason to have absent values in exporters output ? Grafana dashboard is referencing to max(cassandra_stats{datacenter="$datacenter", cluster="$cluster",table=~"$table", name=~"org:apache:cassandra:metrics:table:$keyspace:$table:readlatency:$percentiles"}) by (table), but there is no org:apache:cassandra:metrics:table:* strings at all.

[erebe]

metrics:table:* only exist in cassandra 3.x For cassandra 2.2.x you need to use this path metrics:columnfamily:* that's the old path for table/columnfamily metrics. They contains both the same information.

P.s: You need to remove this line https://github.com/criteo/cassandra_exporter/blob/master/config.yml#L40 in the config file in order to scrap the metrics:columnfamily:* path

[kyxap1]

Already did.

Could you please inspect metrics returned? Looks like they're not usable with current dashboard due absence of all latency/sstable/repair metrics.

cassandra_exporter.metrics.txt commented the string you referenced,

[erebe]

I never tried the dashboard with Cassandra 2.2.x, you certainly need to adapt various metrics path in order for it to work. If you happen to do so, please submit a PR so I can add a version for cassandra 2.2.x also.

Regarding your metrics, at first glance it seems to me that you don't have a lot of them. For a single zipkin node, I export nearly ~2000 metrics, so your ~1200 metrics seems a bit low to me. If you want to know what you are missing, just edit the broken view in the dashboard to see which metric path it is trying to fetch. After that, check that your cassandra_exporter is effectively exporting this metric path.

[kyxap1]

Thank you for explained response.

cassandra_exporter does not output any latency/sstable/repair metrics even with empty blackist in config. But jmx_exporter doing well.

The most reasonable source is wrong objects parser, that's why I have to open the issue.

[erebe]

cassandra_exporter does not output any latency/sstable/repair metrics

You can't expect me to guess by my own which metrics you are talking about.

The log you provided does not talk about any scraping errors regarding sstables/latency/repair. You started this issue only asking for a confirmation of the current behavior for a specific log output. So please understand that I can't help you if you don't give me specific information.

• What is your goal ?
• What issue are you facing regarding achieving this goal ?
• Which metric are you expecting (with the full path) but is not exported ?
• What is your Cassandra exporter configuration ?
• What is your Cassandra setup (version, running inside docker, ...) ?
• Can you provide me a way to reproduce your issue ?

[erebe]

Feel free to re-open with the information :)

[mimarpe]

Hi,

this appears repeatedly in my logs with Cassandra 2.2.10 (will upgrade to 3.x soon):

[main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for com.criteo.nosql.cassandra.exporter.JmxScraper$MBeanInfo@49070868Value javax.management.AttributeNotFoundException: No such attribute: Value at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:81) at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647) at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678) at com.sun.jmx.remote.security.MBeanServerAccessController.getAttribute(MBeanServerAccessController.java:320) at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1445) at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76) at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309) at java.security.AccessController.doPrivileged(Native Method) at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1408) at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:639) at sun.reflect.GeneratedMethodAccessor25.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:357) at sun.rmi.transport.Transport$1.run(Transport.java:200) at sun.rmi.transport.Transport$1.run(Transport.java:197) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.Transport.serviceCall(Transport.java:196) at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:573) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:835) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:688) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.dt_access$417(TCPTransport.java) at java.security.AccessController.doPrivileged(Native Method) at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:687) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) at sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteCall.java:283) at sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:260) at sun.rmi.server.UnicastRef.invoke(UnicastRef.java:161) at com.sun.jmx.remote.internal.PRef.invoke(Unknown Source) at javax.management.remote.rmi.RMIConnectionImpl_Stub.getAttribute(Unknown Source) at javax.management.remote.rmi.RMIConnector$RemoteMBeanServerConnection.getAttribute(RMIConnector.java:903) at com.criteo.nosql.cassandra.exporter.JmxScraper.updateMetric(JmxScraper.java:254) at com.criteo.nosql.cassandra.exporter.JmxScraper.lambda$run$7(JmxScraper.java:164) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270) at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175) at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) at java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1553) at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151) at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174) at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418) at com.criteo.nosql.cassandra.exporter.JmxScraper.run(JmxScraper.java:164) at com.criteo.nosql.cassandra.exporter.Main.main(Main.java:36)

My config is the example one, adding my credentials, so this line appears:

• com:criteo:nosql:cassandra:exporter:.*

Shouldn't that metric be skipped anyway? Also being an internal metric I was surprised it fails.

Besides, what is the cassandra_exporter compatibility with Cassandra versions, also to display stuff in Grafana?

Please let me know, thank you very much.

[erebe]

Hello, It should work with Cassandra 2.2.10, I will try to have a look at it today or tomorrow when I have time. Do you run Cassandra with a special jvm, or some kind of security features enabled ?

Regarding com:criteo:nosql:cassandra:exporter:.*, as you state this metric is internal to the exporter and will never be scrapped anyway. It is just a joke of mine. But in your stack trace, com.criteo.nosql.cassandra.exporter.JmxScraper$MBeanInfo@49070868Value refer just to an MBeanInfo holded by the class JmxScrapper of the exporter, that's not the correct name of the MBeanInfo (should fix that also)

Will try to have a look at it

[mimarpe]

Thanks for your reply. Maybe I missexplained, the metrica are exported correctly actually. I seem to only suffer the exception retriving the value of that MBeanInfo. But what I didn't understand is why are those values from the internal components retrieved?

[erebe]

Hi back,

I have tried the exporter with Cassandra 2.2.10 with openjdk and I don't have any exceptions.

But what I didn't understand is why are those values from the internal components retrieved? It is not an internal components, it just that I made a mistake when rendering the log message. Instead of displaying the name of the metric, I was displaying the object name.

com.criteo.nosql.cassandra.exporter.JmxScraper$MBeanInfo@49070868 can be read as an object MBeanInfo with an internal ID 49070868 holded/owned by the class com.criteo.nosql.cassandra.exporter.JmxScraper

I have fixed the log message in this commit https://github.com/criteo/cassandra_exporter/commit/eae99b72b7dc1a8a9e6ae07adb667f8cafa29aef can you try to run this release of cassandra exporter https://github.com/criteo/cassandra_exporter/releases/download/2.2.2/cassandra_exporter-2.2.1-all.jar in order to know which metric is causing you an exception ?

[mimarpe]

Hi,

thanks for the quick reaction. Understood now about the object vs metric name :-)

Tried it out:

[main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:validationexecutor:currentlyblockedtasks:value Value javax.management.AttributeNotFoundException: No such attribute: Value

According to the content of the MBean using Jconsole, the attribute should be Count instead of Value.

[erebe]

Can you tell me which JVM and version you are using in order to run cassandra ? java -version

[mimarpe]

Hi,

sure:

$ java -version java version "1.8.0_172" Java(TM) SE Runtime Environment (build 1.8.0_172-b11) Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)

But I don't think the metric names and attributes come from the JVM anyway but Cassandra itself, no?

Thanks for looking into this :-)

[erebe]

I think it is related to the cache attributes. Can you run the exporter in verbose mode and dump me the output please java -Dorg.slf4j.simpleLogger.defaultLogLevel=trace -jar cassandra_exporter-2.2.1-all.jar 2>&1 | grep -i 'currentlyblockedtasks'

Thanks

[mimarpe]

Hi,

here you have the extract of the log for the first loop:

$ java -Dorg.slf4j.simpleLogger.defaultLogLevel=trace -jar cassandra_exporter-2.2.1-all.jar config.yml 2>&1 | grep -i 'currentlyblockedtasks' [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 2ms for org:apache:cassandra:metrics:threadpools:request:requestresponsestage:currentlyblockedtasks:value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:validationexecutor:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:memtablereclaimmemory:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:internal:migrationstage:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:pendingrangecalculator:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:miscstage:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:metrics:threadpools:transport:native-transport-requests:currentlyblockedtasks:value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:sampler:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:memtableflushwriter:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:cachecleanupexecutor:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:metrics:threadpools:request:countermutationstage:currentlyblockedtasks:value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:internal:compactionexecutor:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:request:mutationstage:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:cachecleanupexecutor:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:antientropystage:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:migrationstage:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:memtablepostflush:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:metrics:threadpools:request:readstage:currentlyblockedtasks:value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:commitlogarchiver:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:request:readstage:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:miscstage:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:internalresponsestage:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:pendingrangecalculator:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:request:readrepairstage:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:hintedhandoff:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:request:readrepairstage:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:antientropystage:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:antientropysessions:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:internalresponsestage:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:sampler:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:request:requestresponsestage:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:memtablepostflush:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:memtablereclaimmemory:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:metrics:threadpools:request:mutationstage:currentlyblockedtasks:value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:compactionexecutor:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:transport:native-transport-requests:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:antientropysessions:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:request:countermutationstage:currentlyblockedtasks [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:internal:validationexecutor:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:memtableflushwriter:currentlyblockedtasks:value Value [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:commitlogarchiver:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 1ms for org:apache:cassandra:internal:hintedhandoff:currentlyblockedtasks [main] ERROR com.criteo.nosql.cassandra.exporter.JmxScraper - Cannot get value for org:apache:cassandra:metrics:threadpools:internal:gossipstage:currentlyblockedtasks:value Value [main] TRACE com.criteo.nosql.cassandra.exporter.JmxScraper - Scrapping took 0ms for org:apache:cassandra:internal:gossipstage:currentlyblockedtasks

Is this what you needed?

Thanks!

[mimarpe]

And in fact, all the metrics non readable (because of attribute name), attached. no_value.txt

[erebe]

Yes thanks that's perfect. I will take a look at it today. I keep you posted

[erebe]

So I took a look at it, it seems for unknown reason to me that you have some metrics currentlyblockedtasks that have an attribute value instead of count. For example this one org:apache:cassandra:metrics:threadpools:request:requestresponsestage:currentlyblockedtasks:value. You may want to confirm this with jconsole.

The issue it causes is that in the exporter, attributes list are cached, as they are expensive to retrieve, for metrics that have the same name/type/depth (see https://github.com/criteo/cassandra_exporter/blob/master/src/main/java/com/criteo/nosql/cassandra/exporter/JmxScraper.java#L211) In your case, the list of attributes cached conflict/collide between the metrics currentlyblockedtasks that has value as an attribute and the other that has count.

I tried to reproduce the behavior with cassandra 2.2.10, but the official docker image does not expose metrics currentlyblockedtasks with an value attribute, neither all version > 2.2.x that I tried. So in your case I don't know why your cassandra expose those attributes for those metrics.

Anyway if you can confirm that org:apache:cassandra:metrics:threadpools:request:requestresponsestage:currentlyblockedtasks:value expose an value attribute in your case it would be great

[mimarpe]

Hi,

thanks a lot for the quick answers, as usual. Indeed I can confirm that, and I guess you saw the full list of metrics I sent you with attribute name conflicts: https://github.com/criteo/cassandra_exporter/issues/23#issuecomment-460549406

I show proof here with a JConsole screenshot:

I think this might be related indeed with the differences between my Cassandra installation (DSE related, this Cassandra is embedded in a software suite) and the stock one you have...

In any case, I don't think is worth it to dig deeper since these are just a few metrics I can survive without, plus I plan to move to latest 3.x stock Cassandra soon.

According to your code, you are exploring the whole hierarchy of attributes to get their names (and cache them), no? Which I find sensible.

>             ObjectName mbeanName = mBeanInfo.mBeanName;
>             String name = "" + mbeanName.getKeyPropertyList().size() + mbeanName.getKeyProperty("type") + mbeanName.getKeyProperty("name");
>             MBeanAttributeInfo[] info = mBeansAttributesCache.computeIfAbsent(name, xx -> {
>                
>                     return beanConn.getMBeanInfo(mbeanName).getAttributes();

However, how come then a different attribute name is queried? Maybe I am not understanding the purpose of your code just by reading :-)

[erebe]

ok for the switch to the stock Cassandra, in the mean time if you want you can add .*:currentlyblockedtasks:value in the blacklist in order to not scrap them and not have the exception if you don't use those metrics.

Regarding the why, mBeansAttributesCache is here to cache the list of attributes for mbeans because beanConn.getMBeanInfo(mbeanName).getAttributes(); is a very slow jmx calls that take a lot of time as there is a lot of metrics to scrap in Cassandra.

The original hypothesis is that metrics with the same name tends to have the same attributes in the end, so I compute a custom name mbeanName.getKeyPropertyList().size() + mbeanName.getKeyProperty("type") + mbeanName.getKeyProperty("name"); that make several metrics collide in order to avoid having to retrieve the attribute list each time. It is just that in your case the assumption that metrics with the same name have the same attributes list is false ...

[mimarpe]

Ok thanks a lot, understood now. Will do that!