Micrometer-akka for monitoring our AKKA cluster

[DatPhanTien]

Dear all,

We are using AKKA for message handling in our cluster. For monitoring it, we found that Micrometer-akka is one of the emerging and active project on Github.

We just wonder, since we are not familiar with Scala, that how one can export metrics using Micrometer-akka? Whether starting up as an agent amongst starting up our AKKA cluster will be enough? OR we need to implement our own collectors (of course using micrometer-akka API) to export the monitoring metrics?

Thank you in advance for enlighten us.

Best regards Tien Dat

[pjfanning]

Basically all you need is to enable the aspectjweaver java agent and have the micrometer-akka jar in your classpath - https://github.com/kontainers/micrometer-akka#usage

[DatPhanTien]

Dear,

Thanks for your quick response. Does that mean: when we 1) add the java agent and 2) include the micrometer-akka jar into our application JAVA process, the Micrometer-akka monitoring server will be automatically startup along with the startup of AKKA system?

[pjfanning]

I forgot one aspect and have updated https://github.com/kontainers/micrometer-akka/blob/master/README.md#usage (the extra bit is the piece around registering the Meter Registry)

Could you also read https://github.com/kontainers/micrometer-akka/blob/master/README.md#configuration

[DatPhanTien]

I see. That was the reason why it did not work.

Btw, do you have any example in JAVA. We are starting up our AKKA cluster in a JAVA application. (We use the akka.cluster.Cluster.registerOnMemberUp() to startup the AKKA system).

[pjfanning]

We don't have a Java example - the Scala one is all we have)

[DatPhanTien]

I will try my best then. I will update here just in case any one needs an example, if I succeed. By the way, the configuration for Micrometer-akka should be in the application.conf (which is going to be used by AKKA when it starts), isn't it?

[DatPhanTien]

One more question regarding the metrics exporting. Does Micrometer-akka has some metrics exported by default? Or one has to implement into their application to have a metric exported?

A simple example would be perfect. Thanks

[pjfanning]

https://github.com/kontainers/micrometer-akka/blob/master/README.md#configuration has a description of the config (that goes in your application.conf). The micrometer-akka-sample is very easy to run and demonstrates the type of metrics you get.

[DatPhanTien]

Thanks for your answer. I am aware of this. But when I run, the HTTP server returned no metrics. That is why I ask if I miss something or not.

[pjfanning]

I just ran the example as described in its readme and it generated loads of akka metrics.

# HELP akka_router_error_count_webserver_system_io_tcp_selectors_total  
# TYPE akka_router_error_count_webserver_system_io_tcp_selectors_total counter
akka_router_error_count_webserver_system_io_tcp_selectors_total 0.0
# HELP akka_dispatcher_threadpoolexecutor_largest_pool_size  
# TYPE akka_dispatcher_threadpoolexecutor_largest_pool_size gauge
akka_dispatcher_threadpoolexecutor_largest_pool_size{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_largest_pool_size{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 16.0
# HELP akka_dispatcher_forkjoinpool_queued_submission_count  
# TYPE akka_dispatcher_forkjoinpool_queued_submission_count gauge
akka_dispatcher_forkjoinpool_queued_submission_count{dispatcherName="WebServer_akka.actor.default-dispatcher",} 0.0
# HELP akka_dispatcher_threadpoolexecutor_completed_task_count  
# TYPE akka_dispatcher_threadpoolexecutor_completed_task_count gauge
akka_dispatcher_threadpoolexecutor_completed_task_count{dispatcherName="akka.io.pinned-dispatcher",} 0.0
akka_dispatcher_threadpoolexecutor_completed_task_count{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 29.0
# HELP akka_dispatcher_forkjoinpool_parellelism  
# TYPE akka_dispatcher_forkjoinpool_parellelism gauge
akka_dispatcher_forkjoinpool_parellelism{dispatcherName="WebServer_akka.actor.default-dispatcher",} 24.0
# HELP akka_system_actor_count  
# TYPE akka_system_actor_count gauge
akka_system_actor_count{actorSystem="WebServer",} 15.0
# HELP akka_dispatcher_forkjoinpool_pool_size  
# TYPE akka_dispatcher_forkjoinpool_pool_size gauge
akka_dispatcher_forkjoinpool_pool_size{dispatcherName="WebServer_akka.actor.default-dispatcher",} 24.0
# HELP akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_max  
# TYPE akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_max gauge
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_max 0.0025174
# HELP akka_router_routing_time_webserver_system_io_tcp_selectors_seconds  
# TYPE akka_router_routing_time_webserver_system_io_tcp_selectors_seconds histogram
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001",} 1.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001048576",} 1.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001398101",} 1.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001747626",} 1.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002097151",} 1.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002446676",} 2.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002796201",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003145726",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003495251",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003844776",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.004194304",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.005592405",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.006990506",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.008388607",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.009786708",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.011184809",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.01258291",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.013981011",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.015379112",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.016777216",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.022369621",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.027962026",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.033554431",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.039146836",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.044739241",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.050331646",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.055924051",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.061516456",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.067108864",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.089478485",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.111848106",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.134217727",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.156587348",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.178956969",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.20132659",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.223696211",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.246065832",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.268435456",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.357913941",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.447392426",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.536870911",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.626349396",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.715827881",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.805306366",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.894784851",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.984263336",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.073741824",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.431655765",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.789569706",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.147483647",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.505397588",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.863311529",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.22122547",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.579139411",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.937053352",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="4.294967296",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="5.726623061",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="7.158278826",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="8.589934591",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="10.021590356",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="11.453246121",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="12.884901886",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="14.316557651",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="15.748213416",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="17.179869184",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="22.906492245",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="28.633115306",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="30.0",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="+Inf",} 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_count 3.0
akka_router_routing_time_webserver_system_io_tcp_selectors_seconds_sum 0.004812895
# HELP akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_max  
# TYPE akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_max gauge
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_max 0.002742761
# HELP akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds  
# TYPE akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds histogram
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001",} 2.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001048576",} 2.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001398101",} 2.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001747626",} 2.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002097151",} 2.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002446676",} 3.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002796201",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003145726",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003495251",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003844776",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.004194304",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.005592405",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.006990506",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.008388607",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.009786708",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.011184809",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.01258291",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.013981011",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.015379112",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.016777216",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.022369621",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.027962026",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.033554431",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.039146836",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.044739241",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.050331646",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.055924051",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.061516456",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.067108864",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.089478485",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.111848106",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.134217727",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.156587348",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.178956969",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.20132659",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.223696211",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.246065832",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.268435456",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.357913941",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.447392426",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.536870911",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.626349396",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.715827881",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.805306366",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.894784851",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="0.984263336",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="1.073741824",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="1.431655765",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="1.789569706",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="2.147483647",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="2.505397588",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="2.863311529",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="3.22122547",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="3.579139411",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="3.937053352",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="4.294967296",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="5.726623061",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="7.158278826",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="8.589934591",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="10.021590356",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="11.453246121",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="12.884901886",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="14.316557651",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="15.748213416",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="17.179869184",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="22.906492245",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="28.633115306",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="30.0",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_bucket{le="+Inf",} 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_count 4.0
akka_router_time_in_mailbox_webserver_system_io_tcp_selectors_seconds_sum 0.005122879
# HELP akka_router_message_count_webserver_system_io_tcp_selectors_total  
# TYPE akka_router_message_count_webserver_system_io_tcp_selectors_total counter
akka_router_message_count_webserver_system_io_tcp_selectors_total 4.0
# HELP akka_dispatcher_threadpoolexecutor_active_thread_count  
# TYPE akka_dispatcher_threadpoolexecutor_active_thread_count gauge
akka_dispatcher_threadpoolexecutor_active_thread_count{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_active_thread_count{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 0.0
# HELP akka_dispatcher_threadpoolexecutor_core_pool_size  
# TYPE akka_dispatcher_threadpoolexecutor_core_pool_size gauge
akka_dispatcher_threadpoolexecutor_core_pool_size{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_core_pool_size{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 16.0
# HELP akka_dispatcher_threadpoolexecutor_current_pool_size  
# TYPE akka_dispatcher_threadpoolexecutor_current_pool_size gauge
akka_dispatcher_threadpoolexecutor_current_pool_size{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_current_pool_size{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 0.0
# HELP akka_dispatcher_forkjoinpool_queued_task_count  
# TYPE akka_dispatcher_forkjoinpool_queued_task_count gauge
akka_dispatcher_forkjoinpool_queued_task_count{dispatcherName="WebServer_akka.actor.default-dispatcher",} 0.0
# HELP akka_dispatcher_forkjoinpool_steal_count  
# TYPE akka_dispatcher_forkjoinpool_steal_count gauge
akka_dispatcher_forkjoinpool_steal_count{dispatcherName="WebServer_akka.actor.default-dispatcher",} 114.0
# HELP akka_dispatcher_forkjoinpool_active_thread_count  
# TYPE akka_dispatcher_forkjoinpool_active_thread_count gauge
akka_dispatcher_forkjoinpool_active_thread_count{dispatcherName="WebServer_akka.actor.default-dispatcher",} 1.0
# HELP akka_dispatcher_forkjoinpool_running_thread_count  
# TYPE akka_dispatcher_forkjoinpool_running_thread_count gauge
akka_dispatcher_forkjoinpool_running_thread_count{dispatcherName="WebServer_akka.actor.default-dispatcher",} 0.0
# HELP akka_actor_group_actor_count  
# TYPE akka_actor_group_actor_count gauge
akka_actor_group_actor_count{groupName="all",} 3.0
# HELP akka_dispatcher_threadpoolexecutor_max_pool_size  
# TYPE akka_dispatcher_threadpoolexecutor_max_pool_size gauge
akka_dispatcher_threadpoolexecutor_max_pool_size{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_max_pool_size{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 16.0
# HELP akka_dispatcher_threadpoolexecutor_total_task_count  
# TYPE akka_dispatcher_threadpoolexecutor_total_task_count gauge
akka_dispatcher_threadpoolexecutor_total_task_count{dispatcherName="akka.io.pinned-dispatcher",} 1.0
akka_dispatcher_threadpoolexecutor_total_task_count{dispatcherName="akka.stream.default-blocking-io-dispatcher",} 29.0
# HELP akka_router_processing_time_webserver_system_io_tcp_selectors_seconds  
# TYPE akka_router_processing_time_webserver_system_io_tcp_selectors_seconds histogram
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001",} 1.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001048576",} 1.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001398101",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.001747626",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002097151",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002446676",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.002796201",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003145726",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003495251",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.003844776",} 2.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.004194304",} 3.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.005592405",} 3.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.006990506",} 3.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.008388607",} 3.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.009786708",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.011184809",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.01258291",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.013981011",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.015379112",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.016777216",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.022369621",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.027962026",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.033554431",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.039146836",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.044739241",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.050331646",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.055924051",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.061516456",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.067108864",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.089478485",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.111848106",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.134217727",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.156587348",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.178956969",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.20132659",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.223696211",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.246065832",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.268435456",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.357913941",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.447392426",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.536870911",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.626349396",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.715827881",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.805306366",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.894784851",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="0.984263336",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.073741824",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.431655765",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="1.789569706",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.147483647",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.505397588",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="2.863311529",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.22122547",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.579139411",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="3.937053352",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="4.294967296",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="5.726623061",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="7.158278826",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="8.589934591",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="10.021590356",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="11.453246121",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="12.884901886",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="14.316557651",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="15.748213416",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="17.179869184",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="22.906492245",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="28.633115306",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="30.0",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_bucket{le="+Inf",} 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_count 4.0
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_sum 0.015053179
# HELP akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_max  
# TYPE akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_max gauge
akka_router_processing_time_webserver_system_io_tcp_selectors_seconds_max 0.009203125