elastic / apm-agent-java

https://www.elastic.co/guide/en/apm/agent/java/current/index.html
Apache License 2.0
567 stars 321 forks source link

Not seeing transactions in Mule 3.9.3 #1284

Closed eduardo-rdgz closed 4 years ago

eduardo-rdgz commented 4 years ago

Describe the bug

We tried to install the elastic-apm-mule3-agent in our project to monitor Mulesoft flows and calls. However we are not able to see information in the Transactions tab of APM. We are currently using mule 3.9.3 and the mule3 1.16.0 agent.

Adding @ElasticStewart for visibility

Steps to reproduce

  1. Compile and install elastic-apm-mule3-agent v1.16.0 with maven
  2. Add elastic-apm-mule3-agent as a dependency to a maven mule 3.9.3 project
  3. Configure the mule3 agent
  4. Run the mule maven project
  5. Metric data will appear but not Transaction data is shown

Expected behavior

We should be able to see Transaction data

Debug logs

Configuration (got this from the github page) ``` # Elastic Apm Java client properties elastic.apm.log_level=DEBUG elastic.apm.instrument=false elastic.apm.active=true elastic.apm.service_name=localesb elastic.apm.server_urls=https://XXXXXXXXXX.apm.us-east-1.aws.cloud.es.io:443 elastic.apm.secret_token=XXXXXXXXXX elastic.apm.application_packages= ## Mule apm specific properties elastic.apm.mule.capture_input_properties=true elastic.apm.mule.capture_input_properties_regex=http_(.*) elastic.apm.mule.capture_output_properties=true elastic.apm.mule.capture_output_properties_regex=(.*) ```
Mulesoft Logs at startup ``` 2020-07-09 16:33:26.896 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.report.ApmServerHealthChecker - Starting healthcheck to https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/ 2020-07-09 16:33:27.260 [elastic-apm-server-healthcheck] INFO co.elastic.apm.agent.report.ApmServerHealthChecker - Elastic APM server is available: { "build_date": "2019-04-29T11:46:58Z", "build_sha": "79e123dbf8c042b1e3cfa30260910ae7e93939d2", "version": "7.0.1"} 2020-07-09 16:33:27.262 [elastic-apm-server-healthcheck] DEBUG co.elastic.apm.agent.report.ApmServerHealthChecker - APM server https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/ version is: 7.0.1 2020-07-09 16:33:46.904 [Attach Listener] DEBUG co.elastic.apm.agent.impl.payload.SystemInfo - container ID is null 2020-07-09 16:33:46.926 [Attach Listener] INFO co.elastic.apm.agent.util.JmxUtils - Found JVM-specific OperatingSystemMXBean interface: com.sun.management.OperatingSystemMXBean 2020-07-09 16:33:47.067 [Attach Listener] INFO co.elastic.apm.agent.configuration.StartupInfo - Starting Elastic APM 1.16.0 as localesb on Java 1.8.0_131 (Oracle Corporation) Mac OS X 10.14.6 2020-07-09 16:33:47.067 [Attach Listener] DEBUG co.elastic.apm.agent.configuration.StartupInfo - instrument: 'false' (source: Java System Properties) 2020-07-09 16:33:47.068 [Attach Listener] DEBUG co.elastic.apm.agent.configuration.StartupInfo - service_name: 'localesb' (source: Java System Properties) 2020-07-09 16:33:47.068 [Attach Listener] DEBUG co.elastic.apm.agent.configuration.StartupInfo - secret_token: 'XXXX' (source: Java System Properties) 2020-07-09 16:33:47.068 [Attach Listener] DEBUG co.elastic.apm.agent.configuration.StartupInfo - server_urls: 'https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443' (source: Java System Properties) 2020-07-09 16:33:47.068 [Attach Listener] DEBUG co.elastic.apm.agent.configuration.StartupInfo - log_level: 'DEBUG' (source: Java System Properties) 2020-07-09 16:33:47.068 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Reloading configuration from APM Server https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/config/v1/agents 2020-07-09 16:33:47.068 [Attach Listener] WARN co.elastic.apm.agent.configuration.StartupInfo - To enable all features and to increase startup times, please configure application_packages 2020-07-09 16:33:47.081 [Attach Listener] DEBUG co.elastic.apm.agent.impl.circuitbreaker.GCStressMonitor - Registering a heap memory pool (PS Eden Space) for stress monitoring 2020-07-09 16:33:47.081 [Attach Listener] DEBUG co.elastic.apm.agent.impl.circuitbreaker.GCStressMonitor - Registering a heap memory pool (PS Old Gen) for stress monitoring 2020-07-09 16:33:47.082 [Attach Listener] DEBUG co.elastic.apm.agent.impl.circuitbreaker.SystemCpuStressMonitor - Successfully obtained reference to the getSystemCpuLoad method of this JVM's OperatingSystemMXBean implementation 2020-07-09 16:33:47.082 [Attach Listener] DEBUG co.elastic.apm.agent.jmx.JmxMetricTracker - Deferring initialization of JMX metric tracking until capture_jmx_metrics is set. 2020-07-09 16:33:47.082 [Attach Listener] INFO co.elastic.apm.agent.impl.ElasticApmTracer - Tracer switched to RUNNING state 2020-07-09 16:33:47.141 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ElasticApmApiInstrumentation$StartTransactionInstrumentation 2020-07-09 16:33:47.147 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ElasticApmApiInstrumentation$StartTransactionWithRemoteParentInstrumentation 2020-07-09 16:33:47.147 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ElasticApmApiInstrumentation$CurrentTransactionInstrumentation 2020-07-09 16:33:47.147 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ElasticApmApiInstrumentation$CurrentSpanInstrumentation 2020-07-09 16:33:47.147 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ElasticApmApiInstrumentation$CaptureExceptionInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.TransactionInstrumentation$SetUserInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.TransactionInstrumentation$EnsureParentIdInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.TransactionInstrumentation$SetResultInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.TransactionInstrumentation$AddCustomContextInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$SetNameInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$SetTypeInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$SetTypesInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$DoCreateSpanInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$SetStartTimestampInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$EndInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$EndWithTimestampInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$CaptureExceptionInstrumentation 2020-07-09 16:33:47.148 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$GetIdInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$GetTraceIdInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$AddStringLabelInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$AddNumberLabelInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$AddBooleanLabelInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$ActivateInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$IsSampledInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$InjectTraceHeadersInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.CaptureExceptionInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.ApiScopeInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.AbstractSpanInstrumentation$LegacyCaptureExceptionInstrumentation 2020-07-09 16:33:47.149 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$SetNameInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$SetTypeInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$DoCreateSpanInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$EndInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$CaptureExceptionInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$GetIdInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$GetTraceIdInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$AddTagInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$ActivateInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.plugin.api.LegacySpanInstrumentation$IsSampledInstrumentation 2020-07-09 16:33:47.150 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.mule4.Mule4OverrideClassLoaderLookupInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$FinishInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$LogInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$TagInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$SetOperationName 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanInstrumentation$GetTraceContextInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmSpanBuilderInstrumentation$CreateSpanInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$ActivateInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$CurrentSpanInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ScopeManagerInstrumentation$CurrentTraceContextInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ApmScopeInstrumentation 2020-07-09 16:33:47.151 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$BaggageItemsInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$ToTraceIdInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.SpanContextInstrumentation$ToSpanIdInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ExternalSpanContextInstrumentation$ToTraceIdInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ExternalSpanContextInstrumentation$ToSpanIdInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applying instrumentation co.elastic.apm.agent.opentracing.impl.ElasticApmTracerInstrumentation 2020-07-09 16:33:47.152 [Attach Listener] DEBUG co.elastic.apm.agent.bci.ElasticApmAgent - Applied 55 advices 2020-07-09 16:33:47.303 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - This APM Server does not support central configuration. Update to APM Server 7.3+ 2020-07-09 16:33:47.304 [elastic-apm-remote-config-poller] DEBUG co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Scheduling next remote configuration reload in 300s ```
Mulesoft Logs when a REST call occurs (Fragment) ``` [2020-07-09 16:36:10,009] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor post invoke 2020-07-09 16:36:10.009 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.Span - startSpan '' 00-b954c60095b0899572e6043bf7945949-0e81c1061400b963-01 (100b0f18) { 2020-07-09 16:36:10.009 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'organizations-main' 00-b954c60095b0899572e6043bf7945949-b51782ef68d23104-01 (c985b74) (124) 2020-07-09 16:36:10.009 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to '' 00-b954c60095b0899572e6043bf7945949-0e81c1061400b963-01 (100b0f18) (1) 2020-07-09 16:36:10.009 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'organizations-main' 00-b954c60095b0899572e6043bf7945949-b51782ef68d23104-01 (c985b74) (125) [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke that provides original event [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke that provides original event [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke [2020-07-09 16:36:10,010] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke that provides original event [2020-07-09 16:36:10,010] INFO org.mule.api.processor.LoggerMessageProcessor [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - use public https Redis Endpoint [2020-07-09 16:36:10,011] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor post invoke 2020-07-09 16:36:10.011 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.Span - startSpan '' 00-b954c60095b0899572e6043bf7945949-0aa7831e8c77ec1b-01 (13d6757a) { 2020-07-09 16:36:10.011 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'organizations-main' 00-b954c60095b0899572e6043bf7945949-b51782ef68d23104-01 (c985b74) (126) 2020-07-09 16:36:10.011 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to '' 00-b954c60095b0899572e6043bf7945949-0aa7831e8c77ec1b-01 (13d6757a) (1) 2020-07-09 16:36:10.011 [[organizations].scoutingesbapi-httpListenerConfig.worker.03] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'organizations-main' 00-b954c60095b0899572e6043bf7945949-b51782ef68d23104-01 (c985b74) (127) [2020-07-09 16:36:10,011] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke [2020-07-09 16:36:10,011] DEBUG co.elastic.apm.mule.listeners.DomainApplicationListener [] [] [organizations].scoutingesbapi-httpListenerConfig.worker.03 - Received message processor pre invoke that provides original event ```
Mulesoft Logs when idle (Fragment) ``` 2020-07-09 16:36:26.722 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2020-07-09 16:36:26.723 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2020-07-09 16:36:27.151 [elastic-apm-request-timeout-timer] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Request flush because the request timeout occurred 2020-07-09 16:36:27.163 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving FLUSH event (sequence 9) 2020-07-09 16:36:27.163 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Flushing 1447 uncompressed 637 compressed bytes 2020-07-09 16:36:46.925 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICS event (sequence 10) 2020-07-09 16:36:46.925 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Starting new request to https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/intake/v2/events 2020-07-09 16:36:47.080 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Scheduling request timeout in 10s 2020-07-09 16:36:56.722 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2020-07-09 16:36:56.722 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2020-07-09 16:36:57.083 [elastic-apm-request-timeout-timer] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Request flush because the request timeout occurred 2020-07-09 16:36:57.093 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving FLUSH event (sequence 11) 2020-07-09 16:36:57.093 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Flushing 1448 uncompressed 636 compressed bytes 2020-07-09 16:37:16.924 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICS event (sequence 12) 2020-07-09 16:37:16.924 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Starting new request to https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/intake/v2/events 2020-07-09 16:37:17.115 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Scheduling request timeout in 10s 2020-07-09 16:37:26.720 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2020-07-09 16:37:26.721 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2020-07-09 16:37:27.117 [elastic-apm-request-timeout-timer] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Request flush because the request timeout occurred 2020-07-09 16:37:27.125 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving FLUSH event (sequence 13) 2020-07-09 16:37:27.125 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Flushing 1449 uncompressed 650 compressed bytes 2020-07-09 16:37:46.926 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICS event (sequence 14) 2020-07-09 16:37:46.926 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Starting new request to https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/intake/v2/events 2020-07-09 16:37:47.083 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Scheduling request timeout in 10s 2020-07-09 16:37:56.720 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2020-07-09 16:37:56.720 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2020-07-09 16:37:57.086 [elastic-apm-request-timeout-timer] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Request flush because the request timeout occurred 2020-07-09 16:37:57.096 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving FLUSH event (sequence 15) 2020-07-09 16:37:57.097 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Flushing 1449 uncompressed 639 compressed bytes 2020-07-09 16:38:16.925 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving METRICS event (sequence 16) 2020-07-09 16:38:16.925 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Starting new request to https://e6ff862c3d80450f9baf92624dac8f37.apm.us-east-1.aws.cloud.es.io:443/intake/v2/events 2020-07-09 16:38:17.086 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Scheduling request timeout in 10s 2020-07-09 16:38:26.719 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Beginning scheduled configuration reload (interval is 30 sec)... 2020-07-09 16:38:26.719 [elastic-apm-configuration-reloader] DEBUG co.elastic.apm.agent.impl.ElasticApmTracerBuilder - Finished scheduled configuration reload 2020-07-09 16:38:27.089 [elastic-apm-request-timeout-timer] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Request flush because the request timeout occurred 2020-07-09 16:38:27.092 [elastic-apm-server-reporter] DEBUG co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Receiving FLUSH event (sequence 17) ```
eyalkoren commented 4 years ago

@michaelhyatt Can you please advice here?

eyalkoren commented 4 years ago

@eduardo-rdgz Can you see metrics data?

michaelhyatt commented 4 years ago

@eduardo-rdgz the right repo to raise issues against Mule agent is https://github.com/michaelhyatt/elastic-apm-mule3-agent/issues. Really appreciate if you can provide a sample Mule project that helps me reproduce the issue.

BTW, please have a look if this patch addresses the issue: https://github.com/michaelhyatt/elastic-apm-mule3-agent/issues/28

eyalkoren commented 4 years ago

@eduardo-rdgz I am leaving this issue open until you open one in https://github.com/michaelhyatt/elastic-apm-mule3-agent/issues. Once you're done, please make sure to close this one. Thanks!

eduardo-rdgz commented 4 years ago

@eyalkoren Yes we are able to see metrics data. There is a similar issue in the mule3 agent repo. I'll try Michael suggestions and if I'm not succesfull I'll open the ticket there.

@michaelhyatt I'll check the thread you sent to look for potential issues. Also I was thinking the that correct repo should be the mule3 agent, but was instructed to open the bug here.

Thanks!