search

elastic / apm-agent-java

https://www.elastic.co/guide/en/apm/agent/java/current/index.html
Apache License 2.0
567 stars 320 forks source link

apm agent finds server but failed to send data (security manager) #1625

Open cdalexndr opened 3 years ago

cdalexndr commented 3 years ago

Describe the bug

Trying to use apm on elasticsearch jvm to get jvm metrics. APM agent finds APM server, but fails to send any data with java.net.SocketPermission access denied.

Steps to reproduce

elasticapm.properties:

service_name=elasticsearch
server_urls=http://apm:8200

dockercompose.yml:

  elasticsearch:
    image: elasticsearch:7.9.3
    mem_limit: 768m
    expose:
      - "9200"
    environment:
      ES_JAVA_OPTS: -Xmx768m
      HTTP_PORT: 9200
      discovery.type: single-node
      ELASTIC_APM_SERVER_URLS: http://apm:8200
    labels:
      co.elastic.logs/module: elasticsearch
      app: elasticsearch
    volumes:
      - es-data:/usr/share/elasticsearch/data
      - ./config/es/jvm.options:/usr/share/elasticsearch/config/jvm.options:ro
      - ./config/es/apm:/usr/share/elasticsearch/config/apm:ro
  apm:
    image: store/elastic/apm-server:7.9.3
    mem_limit: 64m
    depends_on:
      - elasticsearch
    expose:
      - "8200"
    labels:
      app: apm
    environment:
      apm-server.host: localhost:8200
      output.elasticsearch.hosts: http://elasticsearch:9200
    volumes:
      - apm-data:/usr/share/apm-server/data

Expected behavior

Data sent to apm server.

Debug logs

docker logs docker_elasticsearch_1 | grep apm ``` 2021-01-20 18:17:39,569 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.20.0 as elasticsearch on Java 15 Runtime version: 15+36-1562 VM version: 15+36-1562 (Oracle Corporation) Linux 4.19.128-microsoft-standard 2021-01-20 18:17:39,570 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - VM Arguments: [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-11682935632529430251, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -javaagent:/usr/share/elasticsearch/config/apm/elastic-apm-agent.jar, -Des.cgroups.hierarchy.override=/, -Xmx768m, -XX:MaxDirectMemorySize=402653184, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true] 2021-01-20 18:17:39,571 [main] WARN co.elastic.apm.agent.configuration.StartupInfo - To enable all features and decrease startup time, please configure application_packages 2021-01-20 18:17:41,275 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2020-10-16T08:05:05Z", "build_sha": "af7b123f289d79c34c9da11471803f457eafa5c0", "version": "7.9.3"} 2021-01-20 18:17:41,301 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state {"type": "server", "timestamp": "2021-01-20T18:17:43,849Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "daa16e1cfd08", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-11682935632529430251, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -javaagent:/usr/share/elasticsearch/config/apm/elastic-apm-agent.jar, -Des.cgroups.hierarchy.override=/, -Xmx768m, -XX:MaxDirectMemorySize=402653184, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } 2021-01-20 18:18:11,514 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: access denied ("java.net.SocketPermission" "apm:8200" "connect,resolve") ```
cdalexndr commented 3 years ago

Updated to elastic stack 7.10.1, same issue, server is available but agent isn't sending:

2021-01-21 00:01:26,985 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.20.0 as elasticsearch on Java 15.0.1 Runtime version: 15.0.1+9 VM version: 15.0.1+9 (AdoptOpenJDK) Linux 4.19.128-microsoft-standard
2021-01-21 00:01:26,986 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - VM Arguments: [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-808030813825872369, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -javaagent:/usr/share/elasticsearch/config/apm/elastic-apm-agent.jar, -Des.cgroups.hierarchy.override=/, -Xmx768m, -XX:MaxDirectMemorySize=402653184, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]
2021-01-21 00:01:26,987 [main] WARN  co.elastic.apm.agent.configuration.StartupInfo - To enable all features and decrease startup time, please configure application_packages
2021-01-21 00:01:28,329 [elastic-apm-server-healthcheck] INFO  co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: {  "build_date": "2020-12-04T22:07:34Z",  "build_sha": "b7c209e80c4674603447458e62963ed5246b5297",  "version": "7.10.1"}
2021-01-21 00:01:28,336 [main] INFO  co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state
021-01-21 00:01:58,372 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: access denied ("java.net.SocketPermission" "apm:8200" "connect,resolve")
2021-01-21 00:01:58,373 [elastic-apm-server-reporter] INFO  co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 0 seconds (+/-10%)

Connecting to container, curl works as expected, so it must be an agent connectivity issue.

sh-4.4# curl http://apm:8200                                 
{                                                            
  "build_date": "2020-12-04T22:07:34Z",                      
  "build_sha": "b7c209e80c4674603447458e62963ed5246b5297",   
  "version": "7.10.1"                                        
}
felixbarny commented 3 years ago

Hi and thanks for your issue. Could you share your debug logs as explained here?

cdalexndr commented 3 years ago

APM Agent startup debug logs: (i've added application_packages=none to prevent instrumentation log output)

docker logs docker_elasticsearch_1 | grep apm ``` 2021-01-21 13:25:23,408 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-configuration-reloader` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:23,467 [main] DEBUG co.elastic.apm.agent.impl.payload.SystemInfo - container ID is a5cc4fdcc596a993f7baa86ee0ec861a8d3b33d121fc23f2ca33fbdb48355265 2021-01-21 13:25:23,542 [main] INFO co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean 2021-01-21 13:25:23,611 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-log-shipper` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:23,612 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.20.0 as elasticsearch on Java 15.0.1 Runtime version: 15.0.1+9 VM version: 15.0.1+9 (AdoptOpenJDK) Linux 4.19.128-microsoft-standard 2021-01-21 13:25:23,613 [main] INFO co.elastic.apm.agent.configuration.StartupInfo - VM Arguments: [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2635288539502455485, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -javaagent:/usr/share/elasticsearch/config/apm/elastic-apm-agent.jar, -Des.cgroups.hierarchy.override=/, -Xmx768m, -XX:MaxDirectMemorySize=402653184, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true] 2021-01-21 13:25:23,614 [main] DEBUG co.elastic.apm.agent.configuration.StartupInfo - service_name: 'elasticsearch' (source: /usr/share/elasticsearch/config/apm/elasticapm.properties) 2021-01-21 13:25:23,614 [main] DEBUG co.elastic.apm.agent.configuration.StartupInfo - server_urls: 'http://apm:8200' (source: Environment Variables) 2021-01-21 13:25:23,614 [main] DEBUG co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'none' (source: /usr/share/elasticsearch/config/apm/elasticapm.properties) 2021-01-21 13:25:23,615 [main] DEBUG co.elastic.apm.agent.configuration.StartupInfo - log_level: 'DEBUG' (source: Environment Variables) 2021-01-21 13:25:23,617 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - No plugins dir 2021-01-21 13:25:24,104 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-type-cache-pool-cleaner` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:24,113 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.ApacheHttpClientInstrumentation 2021-01-21 13:25:24,192 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.ApacheHttpAsyncClientInstrumentation 2021-01-21 13:25:24,207 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.ApacheHttpAsyncClientRedirectInstrumentation 2021-01-21 13:25:24,212 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.LegacyApacheHttpClientInstrumentation 2021-01-21 13:25:24,220 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionInstrumentation 2021-01-21 13:25:24,227 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$StartTransactionWithRemoteParentInstrumentation 2021-01-21 13:25:24,235 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentTransactionInstrumentation 2021-01-21 13:25:24,242 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CurrentSpanInstrumentation 2021-01-21 13:25:24,248 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ElasticApmApiInstrumentation$CaptureExceptionInstrumentation 2021-01-21 13:25:24,252 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetUserInstrumentation 2021-01-21 13:25:24,258 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.TransactionInstrumentation$EnsureParentIdInstrumentation 2021-01-21 13:25:24,267 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.TransactionInstrumentation$SetResultInstrumentation 2021-01-21 13:25:24,271 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.TransactionInstrumentation$AddCustomContextInstrumentation 2021-01-21 13:25:24,275 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetNameInstrumentation 2021-01-21 13:25:24,280 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypeInstrumentation 2021-01-21 13:25:24,287 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetTypesInstrumentation 2021-01-21 13:25:24,292 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$DoCreateSpanInstrumentation 2021-01-21 13:25:24,306 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$SetStartTimestampInstrumentation 2021-01-21 13:25:24,311 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndInstrumentation 2021-01-21 13:25:24,320 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$EndWithTimestampInstrumentation 2021-01-21 13:25:24,328 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$CaptureExceptionInstrumentation 2021-01-21 13:25:24,334 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetIdInstrumentation 2021-01-21 13:25:24,344 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$GetTraceIdInstrumentation 2021-01-21 13:25:24,355 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddStringLabelInstrumentation 2021-01-21 13:25:24,361 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddNumberLabelInstrumentation 2021-01-21 13:25:24,365 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$AddBooleanLabelInstrumentation 2021-01-21 13:25:24,371 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$ActivateInstrumentation 2021-01-21 13:25:24,374 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$IsSampledInstrumentation 2021-01-21 13:25:24,386 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$InjectTraceHeadersInstrumentation 2021-01-21 13:25:24,391 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.CaptureExceptionInstrumentation 2021-01-21 13:25:24,395 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.ApiScopeInstrumentation 2021-01-21 13:25:24,400 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.CaptureTransactionInstrumentation 2021-01-21 13:25:24,409 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.CaptureSpanInstrumentation 2021-01-21 13:25:24,423 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.TracedInstrumentation 2021-01-21 13:25:24,436 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.AbstractSpanInstrumentation$LegacyCaptureExceptionInstrumentation 2021-01-21 13:25:24,445 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetNameInstrumentation 2021-01-21 13:25:24,454 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$SetTypeInstrumentation 2021-01-21 13:25:24,458 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$DoCreateSpanInstrumentation 2021-01-21 13:25:24,468 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$EndInstrumentation 2021-01-21 13:25:24,474 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$CaptureExceptionInstrumentation 2021-01-21 13:25:24,490 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetIdInstrumentation 2021-01-21 13:25:24,501 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$GetTraceIdInstrumentation 2021-01-21 13:25:24,508 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$AddTagInstrumentation 2021-01-21 13:25:24,519 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$ActivateInstrumentation 2021-01-21 13:25:24,524 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.pluginapi.LegacySpanInstrumentation$IsSampledInstrumentation 2021-01-21 13:25:24,531 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.asynchttpclient.AbstractAsyncHttpClientInstrumentation$AsyncHttpClientInstrumentation 2021-01-21 13:25:24,538 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.bootdelegation.BootstrapDelegationClassLoaderInstrumentation 2021-01-21 13:25:24,538 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.errorlogging.Slf4jLoggerErrorCapturingInstrumentation 2021-01-21 13:25:24,544 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.errorlogging.Log4j2LoggerErrorCapturingInstrumentation 2021-01-21 13:25:24,550 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grails.GrailsTransactionNameInstrumentation 2021-01-21 13:25:24,554 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.HttpClientInstrumentation 2021-01-21 13:25:24,561 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.HttpClientAsyncInstrumentation 2021-01-21 13:25:24,568 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.httpclient.HttpRequestHeadersInstrumentation 2021-01-21 13:25:24,574 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.concurrent.ExecutorInstrumentation$ExecutorRunnableInstrumentation 2021-01-21 13:25:24,583 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.concurrent.ExecutorInstrumentation$ExecutorCallableInstrumentation 2021-01-21 13:25:24,591 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.concurrent.ExecutorInstrumentation$ExecutorInvokeAnyAllInstrumentation 2021-01-21 13:25:24,602 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.concurrent.ExecutorInstrumentation$ForkJoinPoolInstrumentation 2021-01-21 13:25:24,609 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.concurrent.ForkJoinTaskInstrumentation 2021-01-21 13:25:24,613 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jaxrs.JaxRsTransactionNameInstrumentation 2021-01-21 13:25:24,615 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jaxws.JaxWsTransactionNameInstrumentation 2021-01-21 13:25:24,618 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.ConnectionInstrumentation 2021-01-21 13:25:24,620 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$ExecuteWithQueryInstrumentation 2021-01-21 13:25:24,624 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$ExecuteUpdateWithQueryInstrumentation 2021-01-21 13:25:24,631 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$ExecuteUpdateNoQueryInstrumentation 2021-01-21 13:25:24,637 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$AddBatchInstrumentation 2021-01-21 13:25:24,640 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$ExecuteBatchInstrumentation 2021-01-21 13:25:24,653 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jdbc.StatementInstrumentation$ExecutePreparedStatementInstrumentation 2021-01-21 13:25:24,658 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.jedis.JedisInstrumentation 2021-01-21 13:25:24,659 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.jedis.JedisSpanNameInstrumentation 2021-01-21 13:25:24,659 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce34StartSpanInstrumentation 2021-01-21 13:25:24,659 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce34StopSpanInstrumentation$OnComplete 2021-01-21 13:25:24,660 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce34StopSpanInstrumentation$OnCompleteExceptionally 2021-01-21 13:25:24,660 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce34StopSpanInstrumentation$OnCancel 2021-01-21 13:25:24,660 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce5StartSpanInstrumentation 2021-01-21 13:25:24,661 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce5StopSpanInstrumentation$OnComplete 2021-01-21 13:25:24,661 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce5StopSpanInstrumentation$OnCompleteExceptionally 2021-01-21 13:25:24,661 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.lettuce.Lettuce5StopSpanInstrumentation$OnCancel 2021-01-21 13:25:24,661 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.redis.redisson.RedisConnectionInstrumentation 2021-01-21 13:25:24,662 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jms.JmsMessageProducerInstrumentation$JmsMessageProducerNoDestinationInstrumentation 2021-01-21 13:25:24,668 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jms.JmsMessageProducerInstrumentation$JmsMessageProducerWithDestinationInstrumentation 2021-01-21 13:25:24,673 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jms.JmsMessageConsumerInstrumentation$ReceiveInstrumentation 2021-01-21 13:25:24,679 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jms.JmsMessageConsumerInstrumentation$SetMessageListenerInstrumentation 2021-01-21 13:25:24,683 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jms.JmsMessageListenerInstrumentation 2021-01-21 13:25:24,690 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jsf.JsfLifecycleInstrumentation$JsfLifecycleExecuteInstrumentation 2021-01-21 13:25:24,699 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.jsf.JsfLifecycleInstrumentation$JsfLifecycleRenderInstrumentation 2021-01-21 13:25:24,706 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.hibernatesearch.HibernateSearch5Instrumentation 2021-01-21 13:25:24,711 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.es.restclient.v5_6.ElasticsearchClientSyncInstrumentation 2021-01-21 13:25:24,712 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.es.restclient.v5_6.ElasticsearchClientAsyncInstrumentation 2021-01-21 13:25:24,712 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.es.restclient.v6_4.ElasticsearchClientSyncInstrumentation 2021-01-21 13:25:24,712 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.es.restclient.v6_4.ElasticsearchClientAsyncInstrumentation 2021-01-21 13:25:24,712 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ServerCallListenerInstrumentation$FinalMethodCall 2021-01-21 13:25:24,725 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ServerCallListenerInstrumentation$NonFinalMethodCall 2021-01-21 13:25:24,736 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ServerCallHandlerInstrumentation 2021-01-21 13:25:24,745 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ServerCallInstrumentation 2021-01-21 13:25:24,754 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ChannelInstrumentation 2021-01-21 13:25:24,764 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.grpc.ClientCallImplInstrumentation$Start 2021-01-21 13:25:24,769 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.KafkaProducerInstrumentation 2021-01-21 13:25:24,769 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.KafkaConsumerInstrumentation 2021-01-21 13:25:24,770 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.KafkaProducerHeadersInstrumentation 2021-01-21 13:25:24,770 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.ConsumerRecordsIteratorInstrumentation 2021-01-21 13:25:24,770 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.ConsumerRecordsRecordsInstrumentation 2021-01-21 13:25:24,770 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.kafka.ConsumerRecordsRecordListInstrumentation 2021-01-21 13:25:24,771 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.mongoclient.ConnectionCommandInstrumentation 2021-01-21 13:25:24,773 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.mongoclient.ConnectionInstrumentation 2021-01-21 13:25:24,773 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.okhttp.OkHttpClientInstrumentation 2021-01-21 13:25:24,782 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.okhttp.OkHttp3ClientInstrumentation 2021-01-21 13:25:24,791 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.okhttp.OkHttpClientAsyncInstrumentation 2021-01-21 13:25:24,803 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.okhttp.OkHttp3ClientAsyncInstrumentation 2021-01-21 13:25:24,810 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$FinishInstrumentation 2021-01-21 13:25:24,811 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$LogInstrumentation 2021-01-21 13:25:24,811 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$TagInstrumentation 2021-01-21 13:25:24,811 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$SetOperationName 2021-01-21 13:25:24,812 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$GetTraceContextInstrumentation 2021-01-21 13:25:24,812 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanBuilderInstrumentation$CreateSpanInstrumentation 2021-01-21 13:25:24,812 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$ActivateInstrumentation 2021-01-21 13:25:24,812 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$CurrentSpanInstrumentation 2021-01-21 13:25:24,818 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$CurrentTraceContextInstrumentation 2021-01-21 13:25:24,818 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmScopeInstrumentation 2021-01-21 13:25:24,819 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$BaggageItemsInstrumentation 2021-01-21 13:25:24,819 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$ToTraceIdInstrumentation 2021-01-21 13:25:24,819 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$ToSpanIdInstrumentation 2021-01-21 13:25:24,820 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ExternalSpanContextInstrumentation$ToTraceIdInstrumentation 2021-01-21 13:25:24,820 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ExternalSpanContextInstrumentation$ToSpanIdInstrumentation 2021-01-21 13:25:24,820 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ElasticApmTracerInstrumentation 2021-01-21 13:25:24,820 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.rabbitmq.ChannelInstrumentation$BasicPublish 2021-01-21 13:25:24,830 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.rabbitmq.ChannelInstrumentation$BasicConsume 2021-01-21 13:25:24,840 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.rabbitmq.ChannelInstrumentation$BasicGet 2021-01-21 13:25:24,847 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.ServletInstrumentation 2021-01-21 13:25:24,858 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.ServletVersionInstrumentation$Init 2021-01-21 13:25:24,872 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.ServletVersionInstrumentation$Service 2021-01-21 13:25:24,876 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.FilterChainInstrumentation 2021-01-21 13:25:24,881 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.AsyncInstrumentation$StartAsyncInstrumentation 2021-01-21 13:25:24,884 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.AsyncInstrumentation$AsyncContextInstrumentation 2021-01-21 13:25:24,891 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.servlet.RequestStreamRecordingInstrumentation 2021-01-21 13:25:24,904 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.micrometer.MicrometerInstrumentation 2021-01-21 13:25:24,908 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.resttemplate.SpringRestTemplateInstrumentation 2021-01-21 13:25:24,917 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.springwebmvc.SpringTransactionNameInstrumentation 2021-01-21 13:25:24,920 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.springwebmvc.ServletWrappingControllerServiceNameInstrumentation 2021-01-21 13:25:24,923 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.springwebmvc.ViewRenderInstrumentation 2021-01-21 13:25:24,931 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.springwebmvc.SpringServiceNameInstrumentation 2021-01-21 13:25:24,938 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.springwebmvc.ExceptionHandlerInstrumentation 2021-01-21 13:25:24,941 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.urlconnection.HttpUrlConnectionInstrumentation$CreateSpanInstrumentation 2021-01-21 13:25:24,958 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.urlconnection.HttpUrlConnectionInstrumentation$DisconnectInstrumentation 2021-01-21 13:25:24,961 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.urlconnection.SSLContextInstrumentation 2021-01-21 13:25:24,979 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.scheduled.ScheduledTransactionNameInstrumentation 2021-01-21 13:25:24,993 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.scheduled.TimerTaskInstrumentation 2021-01-21 13:25:25,012 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.quartzjob.JobTransactionNameInstrumentation 2021-01-21 13:25:25,020 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.process.ProcessStartInstrumentation 2021-01-21 13:25:25,023 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.process.ProcessExitInstrumentation$WaitFor 2021-01-21 13:25:25,027 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.process.ProcessExitInstrumentation$Destroy 2021-01-21 13:25:25,030 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.process.CommonsExecAsyncInstrumentation 2021-01-21 13:25:25,037 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applied 151 advices 2021-01-21 13:25:25,104 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation CreateSpanInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class java.net.HttpURLConnection 2021-01-21 13:25:25,110 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ForkJoinTaskInstrumentation: (is(class java.util.concurrent.ForkJoinTask) and not(isInterface())) matches class java.util.concurrent.ForkJoinTask 2021-01-21 13:25:25,113 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ForkJoinPool 2021-01-21 13:25:25,117 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation BootstrapDelegationClassLoaderInstrumentation: ((((((not(name(startsWith(java.))) and not(name(startsWith(jdk.)))) and not(name(startsWith(com.sun.)))) and not(name(startsWith(sun.)))) and not(name(contains(Bootstrap)))) and hasSuperType(erasure(is(class java.lang.ClassLoader)))) and not(isInterface())) matches class co.elastic.apm.agent.bci.classloading.ExternalPluginClassLoader 2021-01-21 13:25:25,303 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$NamedDaemonThreadPoolExecutor 2021-01-21 13:25:25,339 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$1 2021-01-21 13:25:25,339 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ScheduledThreadPoolExecutor 2021-01-21 13:25:25,346 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ThreadPoolExecutor 2021-01-21 13:25:25,346 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.AbstractExecutorService 2021-01-21 13:25:25,484 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation CreateSpanInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class java.net.HttpURLConnection 2021-01-21 13:25:25,485 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation DisconnectInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class java.net.HttpURLConnection 2021-01-21 13:25:25,608 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ForkJoinTaskInstrumentation: (is(class java.util.concurrent.ForkJoinTask) and not(isInterface())) matches class java.util.concurrent.ForkJoinTask 2021-01-21 13:25:25,637 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ForkJoinTaskInstrumentation: ((name(equals(fork)) and returns(erasure(is(class java.util.concurrent.ForkJoinTask)))) and not(isAbstract())) matches public final java.util.concurrent.ForkJoinTask java.util.concurrent.ForkJoinTask.fork() 2021-01-21 13:25:25,694 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ForkJoinPool 2021-01-21 13:25:25,695 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ForkJoinPool 2021-01-21 13:25:25,695 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ForkJoinPool 2021-01-21 13:25:25,695 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ForkJoinPoolInstrumentation: ((hasSuperType(erasure(is(class java.util.concurrent.ForkJoinPool))) and ((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$)))))) and not(isInterface())) matches class java.util.concurrent.ForkJoinPool 2021-01-21 13:25:25,763 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ForkJoinPoolInstrumentation: (((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask)))))) or ((name(equals(submit)) and returns(erasure(is(class java.util.concurrent.ForkJoinTask)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) or ((name(equals(invoke)) and returns(erasure(is(class java.lang.Object)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) and not(isAbstract())) matches public java.lang.Object java.util.concurrent.ForkJoinPool.invoke(java.util.concurrent.ForkJoinTask) 2021-01-21 13:25:25,773 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ForkJoinPoolInstrumentation: (((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask)))))) or ((name(equals(submit)) and returns(erasure(is(class java.util.concurrent.ForkJoinTask)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) or ((name(equals(invoke)) and returns(erasure(is(class java.lang.Object)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) and not(isAbstract())) matches public void java.util.concurrent.ForkJoinPool.execute(java.util.concurrent.ForkJoinTask) 2021-01-21 13:25:25,775 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public void java.util.concurrent.ForkJoinPool.execute(java.lang.Runnable) 2021-01-21 13:25:25,777 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorCallableInstrumentation: ((((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable)))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.ForkJoinTask java.util.concurrent.ForkJoinPool.submit(java.util.concurrent.Callable) 2021-01-21 13:25:25,778 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ForkJoinPoolInstrumentation: (((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask)))))) or ((name(equals(submit)) and returns(erasure(is(class java.util.concurrent.ForkJoinTask)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) or ((name(equals(invoke)) and returns(erasure(is(class java.lang.Object)))) and hasParameter(hasTypes(erasures(containing(is(class java.util.concurrent.ForkJoinTask))))))) and not(isAbstract())) matches public java.util.concurrent.ForkJoinTask java.util.concurrent.ForkJoinPool.submit(java.util.concurrent.ForkJoinTask) 2021-01-21 13:25:25,786 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.ForkJoinTask java.util.concurrent.ForkJoinPool.submit(java.lang.Runnable) 2021-01-21 13:25:25,789 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.ForkJoinTask java.util.concurrent.ForkJoinPool.submit(java.lang.Runnable,java.lang.Object) 2021-01-21 13:25:25,800 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((name(startsWith(invoke)) and (name(endsWith(Any)) or name(endsWith(All)))) and isPublic()) and hasParameter(hasTypes(with(0 matches erasure(is(interface java.util.Collection)))))) and isOverriddenFrom(erasure(is(interface java.util.concurrent.ExecutorService)))) and not(isAbstract())) matches public java.util.List java.util.concurrent.ForkJoinPool.invokeAll(java.util.Collection) 2021-01-21 13:25:25,814 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation BootstrapDelegationClassLoaderInstrumentation: ((((((not(name(startsWith(java.))) and not(name(startsWith(jdk.)))) and not(name(startsWith(com.sun.)))) and not(name(startsWith(sun.)))) and not(name(contains(Bootstrap)))) and hasSuperType(erasure(is(class java.lang.ClassLoader)))) and not(isInterface())) matches class co.elastic.apm.agent.bci.classloading.ExternalPluginClassLoader 2021-01-21 13:25:25,834 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$NamedDaemonThreadPoolExecutor 2021-01-21 13:25:25,835 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$NamedDaemonThreadPoolExecutor 2021-01-21 13:25:25,835 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$NamedDaemonThreadPoolExecutor 2021-01-21 13:25:25,855 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$1 2021-01-21 13:25:25,856 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$1 2021-01-21 13:25:25,856 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class co.elastic.apm.agent.util.ExecutorUtils$1 2021-01-21 13:25:25,888 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ScheduledThreadPoolExecutor 2021-01-21 13:25:25,888 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ScheduledThreadPoolExecutor 2021-01-21 13:25:25,889 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ScheduledThreadPoolExecutor 2021-01-21 13:25:25,929 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public void java.util.concurrent.ScheduledThreadPoolExecutor.execute(java.lang.Runnable) 2021-01-21 13:25:25,931 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.ScheduledThreadPoolExecutor.submit(java.lang.Runnable) 2021-01-21 13:25:25,932 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.ScheduledThreadPoolExecutor.submit(java.lang.Runnable,java.lang.Object) 2021-01-21 13:25:25,933 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorCallableInstrumentation: ((((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable)))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.ScheduledThreadPoolExecutor.submit(java.util.concurrent.Callable) 2021-01-21 13:25:25,939 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.ScheduledFuture java.util.concurrent.ScheduledThreadPoolExecutor.schedule(java.lang.Runnable,long,java.util.concurrent.TimeUnit) 2021-01-21 13:25:25,942 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorCallableInstrumentation: ((((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable)))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.ScheduledFuture java.util.concurrent.ScheduledThreadPoolExecutor.schedule(java.util.concurrent.Callable,long,java.util.concurrent.TimeUnit) 2021-01-21 13:25:25,946 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ThreadPoolExecutor 2021-01-21 13:25:25,947 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ThreadPoolExecutor 2021-01-21 13:25:25,947 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.ThreadPoolExecutor 2021-01-21 13:25:26,010 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public void java.util.concurrent.ThreadPoolExecutor.execute(java.lang.Runnable) 2021-01-21 13:25:26,027 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.AbstractExecutorService 2021-01-21 13:25:26,027 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.AbstractExecutorService 2021-01-21 13:25:26,027 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class java.util.concurrent.AbstractExecutorService 2021-01-21 13:25:26,058 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.AbstractExecutorService.submit(java.lang.Runnable,java.lang.Object) 2021-01-21 13:25:26,062 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorRunnableInstrumentation: ((((((name(equals(execute)) and returns(erasure(is(void)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable)))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable))))))) or ((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(class java.lang.Object))))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.lang.Runnable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.AbstractExecutorService.submit(java.lang.Runnable) 2021-01-21 13:25:26,063 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorCallableInstrumentation: ((((name(equals(submit)) and returns(erasure(hasSuperType(erasure(is(interface java.util.concurrent.Future)))))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable)))))) or ((name(equals(schedule)) and returns(erasure(is(interface java.util.concurrent.ScheduledFuture)))) and hasParameter(hasTypes(erasures(containing(is(interface java.util.concurrent.Callable), is(long), is(class java.util.concurrent.TimeUnit))))))) and not(isAbstract())) matches public java.util.concurrent.Future java.util.concurrent.AbstractExecutorService.submit(java.util.concurrent.Callable) 2021-01-21 13:25:26,064 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((name(startsWith(invoke)) and (name(endsWith(Any)) or name(endsWith(All)))) and isPublic()) and hasParameter(hasTypes(with(0 matches erasure(is(interface java.util.Collection)))))) and isOverriddenFrom(erasure(is(interface java.util.concurrent.ExecutorService)))) and not(isAbstract())) matches public java.util.List java.util.concurrent.AbstractExecutorService.invokeAll(java.util.Collection) throws java.lang.InterruptedException 2021-01-21 13:25:26,066 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((name(startsWith(invoke)) and (name(endsWith(Any)) or name(endsWith(All)))) and isPublic()) and hasParameter(hasTypes(with(0 matches erasure(is(interface java.util.Collection)))))) and isOverriddenFrom(erasure(is(interface java.util.concurrent.ExecutorService)))) and not(isAbstract())) matches public java.util.List java.util.concurrent.AbstractExecutorService.invokeAll(java.util.Collection,long,java.util.concurrent.TimeUnit) throws java.lang.InterruptedException 2021-01-21 13:25:26,068 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((name(startsWith(invoke)) and (name(endsWith(Any)) or name(endsWith(All)))) and isPublic()) and hasParameter(hasTypes(with(0 matches erasure(is(interface java.util.Collection)))))) and isOverriddenFrom(erasure(is(interface java.util.concurrent.ExecutorService)))) and not(isAbstract())) matches public java.lang.Object java.util.concurrent.AbstractExecutorService.invokeAny(java.util.Collection) throws java.lang.InterruptedException,java.util.concurrent.ExecutionException 2021-01-21 13:25:26,069 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((name(startsWith(invoke)) and (name(endsWith(Any)) or name(endsWith(All)))) and isPublic()) and hasParameter(hasTypes(with(0 matches erasure(is(interface java.util.Collection)))))) and isOverriddenFrom(erasure(is(interface java.util.concurrent.ExecutorService)))) and not(isAbstract())) matches public java.lang.Object java.util.concurrent.AbstractExecutorService.invokeAny(java.util.Collection,long,java.util.concurrent.TimeUnit) throws java.lang.InterruptedException,java.util.concurrent.ExecutionException,java.util.concurrent.TimeoutException 2021-01-21 13:25:26,226 [main] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for null containing [co.elastic.apm.agent.bci.classloading.LookupExposer] 2021-01-21 13:25:26,231 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation BootstrapDelegationClassLoaderInstrumentation: ((((((not(name(startsWith(java.))) and not(name(startsWith(jdk.)))) and not(name(startsWith(com.sun.)))) and not(name(startsWith(sun.)))) and not(name(contains(Bootstrap)))) and hasSuperType(erasure(is(class java.lang.ClassLoader)))) and not(isInterface())) matches class co.elastic.apm.agent.bci.classloading.IndyPluginClassLoader 2021-01-21 13:25:26,257 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-server-healthcheck` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,264 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-remote-config-poller` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,274 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation CreateSpanInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class sun.net.www.protocol.http.HttpURLConnection 2021-01-21 13:25:26,281 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation DisconnectInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class sun.net.www.protocol.http.HttpURLConnection 2021-01-21 13:25:26,376 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation CreateSpanInstrumentation: ((((name(equals(connect)) and hasParameter(ofSize(0))) or (name(equals(getOutputStream)) and hasParameter(ofSize(0)))) or (name(equals(getInputStream)) and hasParameter(ofSize(0)))) and not(isAbstract())) matches public void sun.net.www.protocol.http.HttpURLConnection.connect() throws java.io.IOException 2021-01-21 13:25:26,378 [main] DEBUG co.elastic.apm.agent.impl.circuitbreaker.GCStressMonitor - Registering a heap memory pool (Tenured Gen) for stress monitoring 2021-01-21 13:25:26,380 [main] DEBUG co.elastic.apm.agent.impl.circuitbreaker.GCStressMonitor - Registering a heap memory pool (Eden Space) for stress monitoring 2021-01-21 13:25:26,388 [main] DEBUG co.elastic.apm.agent.impl.circuitbreaker.SystemCpuStressMonitor - Successfully obtained reference to the getSystemCpuLoad method of this JVM's OperatingSystemMXBean implementation 2021-01-21 13:25:26,392 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-circuit-breaker` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,402 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-weak-map-cleaner` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,414 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-shared` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,455 [main] DEBUG co.elastic.apm.agent.jmx.JmxMetricTracker - Deferring initialization of JMX metric tracking until capture_jmx_metrics is set. 2021-01-21 13:25:26,456 [main] DEBUG co.elastic.apm.agent.util.ExecutorUtils - A new thread named `elastic-apm-sampling-profiler` was created. The original context class loader of this thread (jdk.internal.loader.ClassLoaders$AppClassLoader@3764951d) has been overridden 2021-01-21 13:25:26,515 [main] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state 2021-01-21 13:25:26,557 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation CreateSpanInstrumentation: ((((name(equals(connect)) and hasParameter(ofSize(0))) or (name(equals(getOutputStream)) and hasParameter(ofSize(0)))) or (name(equals(getInputStream)) and hasParameter(ofSize(0)))) and not(isAbstract())) matches public synchronized java.io.OutputStream sun.net.www.protocol.http.HttpURLConnection.getOutputStream() throws java.io.IOException 2021-01-21 13:25:26,559 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation CreateSpanInstrumentation: ((((name(equals(connect)) and hasParameter(ofSize(0))) or (name(equals(getOutputStream)) and hasParameter(ofSize(0)))) or (name(equals(getInputStream)) and hasParameter(ofSize(0)))) and not(isAbstract())) matches public synchronized java.io.InputStream sun.net.www.protocol.http.HttpURLConnection.getInputStream() throws java.io.IOException 2021-01-21 13:25:26,566 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation DisconnectInstrumentation: ((name(equals(disconnect)) and hasParameter(ofSize(0))) and not(isAbstract())) matches public void sun.net.www.protocol.http.HttpURLConnection.disconnect() 2021-01-21 13:25:26,602 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation CreateSpanInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class javax.net.ssl.HttpsURLConnection 2021-01-21 13:25:26,602 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation DisconnectInstrumentation: ((hasSuperType(erasure(is(class java.net.HttpURLConnection))) and not(name(equals(sun.net.www.protocol.https.HttpsURLConnectionImpl)))) and not(isInterface())) matches class javax.net.ssl.HttpsURLConnection 2021-01-21 13:25:26,666 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Reloading configuration from APM Server http://apm:8200/config/v1/agents 2021-01-21 13:25:26,667 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.report.ApmServerHealthChecker - Starting healthcheck to http://apm:8200/ 2021-01-21 13:25:26,752 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for null containing [co.elastic.apm.agent.urlconnection.HttpUrlConnectionInstrumentation$CreateSpanInstrumentation, co.elastic.apm.agent.urlconnection.SSLContextInstrumentation, co.elastic.apm.agent.urlconnection.package-info, co.elastic.apm.agent.urlconnection.HttpUrlConnectionInstrumentation$DisconnectInstrumentation, co.elastic.apm.agent.urlconnection.UrlConnectionPropertyAccessor, co.elastic.apm.agent.urlconnection.HttpUrlConnectionInstrumentation, co.elastic.apm.agent.bci.classloading.LookupExposer] 2021-01-21 13:25:26,908 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.apache.logging.log4j.status.StatusLogger 2021-01-21 13:25:26,960 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Central configuration is disabled. Set kibana.enabled: true in your APM Server configuration. 2021-01-21 13:25:26,967 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Scheduling next remote configuration reload in 300s 2021-01-21 13:25:26,968 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2020-12-04T22:07:34Z", "build_sha": "b7c209e80c4674603447458e62963ed5246b5297", "version": "7.10.1"} 2021-01-21 13:25:27,030 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.report.ApmServerHealthChecker - APM server http://apm:8200/ version is: 7.10.1 2021-01-21 13:25:27,036 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.apache.logging.log4j.spi.AbstractLogger 2021-01-21 13:25:27,106 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((name(equals(error)) and (hasParameter(hasTypes(with(0 matches erasure(name(equals(java.lang.String)))))) and hasParameter(hasTypes(with(1 matches erasure(name(equals(java.lang.Throwable)))))))) and not(isAbstract())) matches public void org.apache.logging.log4j.spi.AbstractLogger.error(java.lang.String,java.lang.Throwable) 2021-01-21 13:25:27,173 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.apache.logging.log4j.simple.SimpleLogger 2021-01-21 13:25:28,014 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.elasticsearch.common.logging.PrefixLogger 2021-01-21 13:25:28,021 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.apache.logging.log4j.spi.ExtendedLoggerWrapper 2021-01-21 13:25:28,533 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Log4j2LoggerErrorCapturingInstrumentation: ((hasSuperType(erasure(name(equals(org.apache.logging.log4j.Logger)))) and not(hasSuperType(erasure(name(equals(org.slf4j.Logger)))))) and not(isInterface())) matches class org.apache.logging.log4j.core.Logger 2021-01-21 13:25:28,595 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ProcessStartInstrumentation: (name(equals(java.lang.ProcessBuilder)) and not(isInterface())) matches class java.lang.ProcessBuilder 2021-01-21 13:25:28,600 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation ProcessStartInstrumentation: ((name(equals(start)) and hasParameter(ofSize(0))) and not(isAbstract())) matches public java.lang.Process java.lang.ProcessBuilder.start() throws java.io.IOException 2021-01-21 13:25:28,606 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation WaitFor: ((name(equals(java.lang.ProcessImpl)) or name(equals(java.lang.UNIXProcess))) and not(isInterface())) matches class java.lang.ProcessImpl 2021-01-21 13:25:28,607 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation Destroy: ((name(equals(java.lang.ProcessImpl)) or name(equals(java.lang.UNIXProcess))) and not(isInterface())) matches class java.lang.ProcessImpl 2021-01-21 13:25:28,622 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation WaitFor: (name(equals(waitFor)) and not(isAbstract())) matches public synchronized int java.lang.ProcessImpl.waitFor() throws java.lang.InterruptedException 2021-01-21 13:25:28,623 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation WaitFor: (name(equals(waitFor)) and not(isAbstract())) matches public synchronized boolean java.lang.ProcessImpl.waitFor(long,java.util.concurrent.TimeUnit) throws java.lang.InterruptedException 2021-01-21 13:25:28,624 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation Destroy: ((isPublic() and (name(equals(destroy)) or name(equals(destroyForcibly)))) and not(isAbstract())) matches public void java.lang.ProcessImpl.destroy() 2021-01-21 13:25:28,625 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Method match for instrumentation Destroy: ((isPublic() and (name(equals(destroy)) or name(equals(destroyForcibly)))) and not(isAbstract())) matches public java.lang.Process java.lang.ProcessImpl.destroyForcibly() 2021-01-21 13:25:28,657 [main] DEBUG co.elastic.apm.agent.bci.IndyPluginClassLoaderFactory - Creating plugin class loader for null containing [co.elastic.apm.agent.process.CommonsExecAsyncInstrumentation, co.elastic.apm.agent.process.ProcessHelper, co.elastic.apm.agent.process.BaseProcessInstrumentation, co.elastic.apm.agent.process.ProcessStartInstrumentation$ProcessBuilderStartAdvice, co.elastic.apm.agent.process.package-info, co.elastic.apm.agent.process.ProcessExitInstrumentation$WaitFor$WaitForAdvice, co.elastic.apm.agent.process.ProcessExitInstrumentation$WaitFor, co.elastic.apm.agent.process.ProcessExitInstrumentation$Destroy$DestroyAdvice, co.elastic.apm.agent.process.CommonsExecAsyncInstrumentation$CommonsExecAdvice, co.elastic.apm.agent.process.ProcessStartInstrumentation, co.elastic.apm.agent.process.ProcessExitInstrumentation$Destroy, co.elastic.apm.agent.process.ProcessExitInstrumentation, co.elastic.apm.agent.bci.classloading.LookupExposer] {"type": "server", "timestamp": "2021-01-21T13:25:29,106Z", "level": "INFO", "component": "o.e.n.Node", "cluster.name": "docker-cluster", "node.name": "a5cc4fdcc596", "message": "JVM arguments [-Xshare:auto, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -XX:+ShowCodeDetailsInExceptionMessages, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.locale.providers=SPI,COMPAT, -Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dio.netty.allocator.numDirectArenas=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch-2635288539502455485, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=logs/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -javaagent:/usr/share/elasticsearch/config/apm/elastic-apm-agent.jar, -Des.cgroups.hierarchy.override=/, -Xmx768m, -XX:MaxDirectMemorySize=402653184, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/usr/share/elasticsearch/config, -Des.distribution.flavor=default, -Des.distribution.type=docker, -Des.bundled_jdk=true]" } 2021-01-21 13:25:29,298 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation BootstrapDelegationClassLoaderInstrumentation: ((((((not(name(startsWith(java.))) and not(name(startsWith(jdk.)))) and not(name(startsWith(com.sun.)))) and not(name(startsWith(sun.)))) and not(name(contains(Bootstrap)))) and hasSuperType(erasure(is(class java.lang.ClassLoader)))) and not(isInterface())) matches class org.elasticsearch.plugins.ExtendedPluginsClassLoader 2021-01-21 13:25:29,311 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.plugins.ExtendedPluginsClassLoader at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Inlining$Resolved$ForMethodExit.of(Advice.java:8344) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Inlining.asMethodExit(Advice.java:7606) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:30,031 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation SSLContextInstrumentation: (((name(equals(javax.net.ssl.SSLContext)) or name(equals(javax.net.SocketFactory))) or name(equals(javax.net.ssl.SSLSocketFactory))) and not(isInterface())) matches class javax.net.ssl.SSLContext 2021-01-21 13:25:30,034 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation javax.net.ssl.SSLContext at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithDiscardedEnterType.(Advice.java:9550) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:33,494 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor 2021-01-21 13:25:33,494 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor 2021-01-21 13:25:33,494 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor 2021-01-21 13:25:33,532 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithRetainedEnterType.(Advice.java:9520) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:33,535 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor 2021-01-21 13:25:33,535 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor 2021-01-21 13:25:33,536 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor 2021-01-21 13:25:33,560 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.common.util.concurrent.QueueResizingEsThreadPoolExecutor at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithRetainedEnterType.(Advice.java:9520) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:33,564 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService 2021-01-21 13:25:33,564 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService 2021-01-21 13:25:33,564 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService 2021-01-21 13:25:33,567 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.common.util.concurrent.EsExecutors$DirectExecutorService at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithRetainedEnterType.(Advice.java:9520) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:33,579 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.threadpool.Scheduler$SafeScheduledThreadPoolExecutor 2021-01-21 13:25:33,579 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.threadpool.Scheduler$SafeScheduledThreadPoolExecutor 2021-01-21 13:25:33,579 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.threadpool.Scheduler$SafeScheduledThreadPoolExecutor 2021-01-21 13:25:33,583 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.threadpool.Scheduler$SafeScheduledThreadPoolExecutor at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithRetainedEnterType.(Advice.java:9520) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:33,595 [main] INFO co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - java.util.concurrent.RejectedExecutionException refers to a missing class. 2021-01-21 13:25:33,595 [main] DEBUG co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - ByteBuddy type resolution stack trace: at co.elastic.apm.agent.shaded.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:157) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$DescriptionStrategy$Default$2.apply(AgentBuilder.java:3524) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10344) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:38,538 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation SSLContextInstrumentation: (((name(equals(javax.net.ssl.SSLContext)) or name(equals(javax.net.SocketFactory))) or name(equals(javax.net.ssl.SSLSocketFactory))) and not(isInterface())) matches class javax.net.ssl.SSLSocketFactory 2021-01-21 13:25:38,541 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation javax.net.ssl.SSLSocketFactory at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithDiscardedEnterType.(Advice.java:9550) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:38,544 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation SSLContextInstrumentation: (((name(equals(javax.net.ssl.SSLContext)) or name(equals(javax.net.SocketFactory))) or name(equals(javax.net.ssl.SSLSocketFactory))) and not(isInterface())) matches class javax.net.SocketFactory 2021-01-21 13:25:38,547 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation javax.net.SocketFactory at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithDiscardedEnterType.(Advice.java:9550) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:44,280 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorRunnableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor 2021-01-21 13:25:44,280 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorCallableInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor 2021-01-21 13:25:44,280 [main] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Type match for instrumentation ExecutorInvokeAnyAllInstrumentation: (((((((((hasSuperType(erasure(name(equals(java.util.concurrent.Executor)))) and not(name(equals(org.apache.felix.resolver.ResolverImpl$DumbExecutor)))) and not(name(contains(jetty)))) and not(name(contains(tomcat)))) and not(name(contains(jboss)))) and not(name(contains(undertow)))) and not(name(contains(netty)))) and not(name(startsWith(com.hazelcast)))) and not((name(contains($Proxy)) or name(contains($$))))) and not(isInterface())) matches class org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor 2021-01-21 13:25:44,283 [main] WARN co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - Error on transformation org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$ForMissingType.resolve(AnnotationValue.java:2342) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.description.annotation.AnnotationValue$AbstractBase.resolve(AnnotationValue.java:575) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.(Advice.java:9386) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter$WithRetainedEnterType.(Advice.java:9520) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating$Resolved$ForMethodEnter.of(Advice.java:9409) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$Dispatcher$Delegating.asMethodEnter(Advice.java:8906) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice.to(Advice.java:356) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.asm.Advice$WithCustomMapping.to(Advice.java:11567) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice$Entry$ForUnifiedAdvice.resolve(AgentBuilder.java:2543) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Transformer$ForAdvice.transform(AgentBuilder.java:2329) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10369) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:44,910 [elasticsearch[a5cc4fdcc596][clusterApplierService#updateTask][T#1]] INFO co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - java.util.TreeMap$NavigableSubMap$EntrySetView refers to a missing class. 2021-01-21 13:25:44,910 [elasticsearch[a5cc4fdcc596][clusterApplierService#updateTask][T#1]] DEBUG co.elastic.apm.agent.bci.bytebuddy.ErrorLoggingListener - ByteBuddy type resolution stack trace: at co.elastic.apm.agent.shaded.bytebuddy.pool.TypePool$Resolution$Illegal.resolve(TypePool.java:157) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$DescriptionStrategy$Default$2.apply(AgentBuilder.java:3524) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.doTransform(AgentBuilder.java:10344) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10307) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.access$1600(AgentBuilder.java:10073) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10766) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$Java9CapableVmDispatcher.run(AgentBuilder.java:10704) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer.transform(AgentBuilder.java:10263) ~[?:?] at co.elastic.apm.agent.shaded.bytebuddy.agent.builder.AgentBuilder$Default$ExecutingTransformer$ByteBuddy$ModuleSupport.transform(Unknown Source) ~[?:?] 2021-01-21 13:25:53,407 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2021-01-21 13:25:53,413 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2021-01-21 13:25:58,158 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving JSON_WRITER event (sequence 0) 2021-01-21 13:25:58,175 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Starting new request to http://apm:8200/intake/v2/events 2021-01-21 13:25:58,235 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: access denied ("java.net.SocketPermission" "apm:8200" "connect,resolve") 2021-01-21 13:25:58,236 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Event handling failure java.security.AccessControlException: access denied ("java.net.SocketPermission" "apm:8200" "connect,resolve") at co.elastic.apm.agent.report.AbstractIntakeApiHandler.startRequest(AbstractIntakeApiHandler.java:119) ~[?:?] at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.startRequest(IntakeV2ReportingEventHandler.java:153) ~[?:?] at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.handleEvent(IntakeV2ReportingEventHandler.java:95) ~[?:?] at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.onEvent(IntakeV2ReportingEventHandler.java:73) ~[?:?] at co.elastic.apm.agent.report.IntakeV2ReportingEventHandler.onEvent(IntakeV2ReportingEventHandler.java:44) ~[?:?] at co.elastic.apm.agent.shaded.lmax.disruptor.BatchEventProcessor.processEvents(BatchEventProcessor.java:168) ~[?:?] at co.elastic.apm.agent.shaded.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:125) ~[?:?] 2021-01-21 13:25:58,273 [elastic-apm-server-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 0 seconds (+/-10%) ```

http://apm:8200/intake/v2/events is accesible:

λ docker exec -it docker_elasticsearch_1 sh
sh-4.4# curl http://apm:8200/intake/v2/events
{
  "accepted": 0,
  "errors": [
    {
      "message": "only POST requests are supported"
    }
  ]
}
SylvainJuge commented 3 years ago

Hi @cdalexndr , are you using a security manager in your application ?

The following line in the log seems to indicate that the agent does not have permission to open a socket to reach apm-server.

java.security.AccessControlException: access denied ("java.net.SocketPermission" "apm:8200" "connect,resolve")

If that's the case, you will have to modify the security policy of your setup to include configuration for the agent:

grant codeBase "file:/path/to/agent/elastic-agent.jar" {
  permission java.security.AllPermission;
};

I have to admit that it's something probably missing from our documentation.

eyalkoren commented 3 years ago

I recall reports where specifying the explicit jar did not work for some reason, so if the above is not working, please try this as well:

grant codeBase "file:/path/to/agent/-" {
     permission java.security.AllPermission;
};

And eventually please report back if either solved your problem.

cdalexndr commented 3 years ago

Indeed, adding permission to /usr/share/elasticsearch/jdk/lib/security/default.policy resolves this issue.

Leaving the issue open for documentation update.

eyalkoren commented 3 years ago

Please provide the exact grant codeBase ... clause you used.

SylvainJuge commented 3 years ago

@eyalkoren do you think it would be possible to make the agent detect when a security manager is used and output a snippet to allow end-user to fix it themselves ? Would be more efficient than documentation alone.

eyalkoren commented 3 years ago

Yeah, great idea, we can create this snippet with the actual agent location, which is very useful! 👍