Report ActiveMQ Artemis metrics using Telegraf jolokia2_agent plugin

[ipajumets]

Please direct all support questsions to slack or the forums. Thank you.

So we recently migrated from ActiveMQ "Classic" to ActiveMQ Artemis and it seems like we are unable to get our Telegraf configuration right for inputs.jolokia2_agent.

[[inputs.jolokia2_agent]]
  urls = [ "http://<artemis-live-instance>:8161/console/jolokia/", "http://<artemis-backup-instance>:8161/console/jolokia/" ]
  username = "<user>"
  password = "<password>"

  [[inputs.jolokia2_agent.metric]]
    name  = "artemis"
    mbean = "org.apache.activemq.artemis:type=Broker,brokerName=*,destinationType=*,destinationName=<destination-root>.*"
    paths = [ "AverageEnqueueTime", "AverageMessageSize", "BlockedSends", "ConsumerCount", "DequeueCount", "DispatchCount", "EnqueueCount", "ExpiredCount", "ForwardCount", "InFlightCount", "MaxEnqueueTime", "MemoryPercentUsage", "MemoryUsageByteCount", "MinEnqueueTime", "ProducerCount", "QueueSize", "StoreMessageSize" ]
    tag_keys = [ "destinationName", "destinationType" ]

  [[inputs.jolokia2_agent.metric]]
    name  = "artemis"
    mbean = "org.apache.activemq.artemis:type=Broker,brokerName=*,destinationType=Queue,destinationName=ActiveMQ.DLQ"
    paths = [ "AverageEnqueueTime", "AverageMessageSize", "BlockedSends", "ConsumerCount", "DequeueCount", "DispatchCount", "EnqueueCount", "ExpiredCount", "ForwardCount", "InFlightCount", "MaxEnqueueTime", "MemoryPercentUsage", "MemoryUsageByteCount", "MinEnqueueTime", "ProducerCount", "QueueSize", "StoreMessageSize" ]
    tag_keys = [ "destinationName", "destinationType" ]

We are heavily using Grafana so it's pretty important to get metrics for ActiveMQ Artemis.

[telegraf-tiger[bot]]

Hello! I recommend posting this question in our Community Slack or Community Forums, we have a lot of talented community members there who could help answer your question more quickly. You can also learn more about Telegraf by enrolling at InfluxDB University for free!

Heads up, this issue will be automatically closed after 7 days of inactivity. Thank you!

[ipajumets]

Hello! I recommend posting this question in our Community Slack or Community Forums, we have a lot of talented community members there who could help answer your question more quickly. You can also learn more about Telegraf by enrolling at InfluxDB University for free!

Heads up, this issue will be automatically closed after 7 days of inactivity. Thank you!

Community Slack link seems to be inactive.

[ipajumets]

We were able to get Telegraf to start sending Artemis metrics to InfluxDB.

[[inputs.jolokia2_agent]]
  urls = [ "http://<artemis-live-instance>:8161/console/jolokia/", "http://<artemis-backup-instance>:8161/console/jolokia/" ]
  username = "<user>"
  password = "<password>"

  [[inputs.jolokia2_agent.metric]]
    name  = "artemis"
    mbean = "org.apache.activemq.artemis:broker=\"*\",component=addresses,address=\"<queue-prefix>.*\",subcomponent=queues,routing-type=\"anycast\",queue=\"<queue-prefix>.*\""
    tag_keys = ["queue", "subcomponent"]

But now there's another problem - as you can see I've converted queue names into tags but they are stored with double quotes around them. For example: "example.queue.name" should be just example.queue.name (without double quotes).

[srebhan]

@ipajumets sorry for the late replay... As a workaround you could just trim the quotes for now

[[processors.strings]]
  namepass = ["artemis"]
  [[processors.strings.trim]]
    tag = "queue"
    cutset = '"'

But I agree that we should trim those by default...

[powersj]

Looking at the example directory, there are examples where the mbean using a wildcard does not need quotes around it. Was this something specific for Artemis?

[ipajumets]

@powersj it seems specific to Artemis because there was no such problem with ActiveMQ "Classic".

Also I think it makes sense to add ActiveMQ Artemis examples under jolokia2_agent as it's quite different from ActiveMQ "Classic".

[powersj]

Agreed, would you be willing to put up a PR with your config that you got working?

Thanks!

[ipajumets]

Will do!

[srebhan]

@ipajumets started to implement a test but struggled to extract any metric, but maybe the code gets you kick-started:

func TestIntegrationArtemis(t *testing.T) {
    if testing.Short() {
        t.Skip("Skipping integration test in short mode")
    }

    // Start the docker container
    container := testutil.Container{
        Image:        "apache/activemq-artemis",
        ExposedPorts: []string{"61616", "8161"},
        WaitingFor: wait.ForAll(
            wait.ForLog("Artemis Console available at"),
            wait.ForListeningPort(nat.Port("8161")),
        ),
    }
    require.NoError(t, container.Start(), "failed to start container")
    defer container.Terminate()

    // Setup the plugin
    port := container.Ports["8161"]
    plugin := &jolokia2_agent.JolokiaAgent{
        URLs:     []string{"http://" + container.Address + ":" + port + "/console/jolokia/"},
        Username: "artemis",
        Password: "artemis",
        Metrics: []common.MetricConfig{
            {
                Name:    "artemis",
                Mbean:   `org.apache.activemq.artemis:broker="*",component=addresses,address="<queue-prefix>.*",subcomponent=queues,routing-type="anycast",queue="<queue-prefix>.*"`,
                TagKeys: []string{"queue", "subcomponent"},
            },
        },
    }

    // Setup the expectations
    expected := []telegraf.Metric{
        metric.New(
            "artemis",
            map[string]string{
                "queue":          "somequeue",
                "subcomponent":   "somesubcomponent",
            },
            map[string]interface{}{
                "value": int64(0),
            },
            time.Unix(0, 0),
        ),
    }

    // Collect the metrics and compare
    var acc testutil.Accumulator
    require.NoError(t, plugin.Gather(&acc))

    actual := acc.GetTelegrafMetrics()
    testutil.RequireMetricsStructureEqual(t, expected, actual, testutil.IgnoreTime())
}

I guess there is something else needed to produce the metrics!?

[srebhan]

@ipajumets I found my mistake in the query. Can you please test the binary in PR #14132 available once CI finished the tests successfully!? Let me know if the PR fixes the issue!