Statsd output failure when creating the ActorSystem in AkkaIOStatsdCounterInterface

[vonnagy]

When an application is started the aspects are weaved into the newly created ActorSystem. During that process the AkkaIOStatsdCounterInterface is created which in turn creates a new actor system called "statsd".

At that time the creation of the new system ("statsd") fails when trying to publish a message that "StandardOutLogger started" Logging.scala line 88.

I believe this is due to the aspects being defined as a singleton, but I am not entirely sure. The exception is given below.

[AppClassLoader@3466fcd5] weaveinfo Join point 'method-execution(void akka.event.EventStream.publish(java.lang.Object))' in Type 'akka.event.EventStream' (EventStream.scala:26) advised by before advice from 'org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect' (ActorCellMonitoringAspect.aj:201)
The actor system could not be started: Exception while initializing org_eigengo_monitor_agent_akka_ActorCellMonitoringAspect: org.aspectj.lang.NoAspectBoundException: org_eigengo_monitor_agent_akka_ActorCellMonitoringAspect
org.aspectj.lang.NoAspectBoundException: Exception while initializing org_eigengo_monitor_agent_akka_ActorCellMonitoringAspect: org.aspectj.lang.NoAspectBoundException: org_eigengo_monitor_agent_akka_ActorCellMonitoringAspect
    at org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect.aspectOf(ActorCellMonitoringAspect.aj:1)
    at akka.event.EventStream.publish(EventStream.scala:26)
    at akka.event.LoggingBus$class.startStdoutLogger(Logging.scala:88)
    at akka.event.EventStream.startStdoutLogger(EventStream.scala:26)
    at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:537)
    at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
    at akka.actor.ActorSystem$.create(ActorSystem.scala:73)
    at akka.HarnessActorSystem$.apply(HarnessActorSystem.scala:34)
    at com.webtrends.portfolio.service.HarnessServiceSystem$.com$webtrends$portfolio$service$HarnessServiceSystem$$startActorSystem(HarnessService.scala:86)
    at com.webtrends.portfolio.service.HarnessServiceSystem$delayedInit$body.apply(HarnessService.scala:33)
    at scala.Function0$class.apply$mcV$sp(Function0.scala:40)
    at scala.runtime.AbstractFunction0.apply$mcV$sp(AbstractFunction0.scala:12)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.App$$anonfun$main$1.apply(App.scala:71)
    at scala.collection.immutable.List.foreach(List.scala:318)
    at scala.collection.generic.TraversableForwarder$class.foreach(TraversableForwarder.scala:32)
    at scala.App$class.main(App.scala:71)
    at com.webtrends.portfolio.service.HarnessServiceSystem$.main(HarnessService.scala:22)
    at com.webtrends.portfolio.service.HarnessServiceSystem.main(HarnessService.scala)
Caused by: org.aspectj.lang.NoAspectBoundException: org_eigengo_monitor_agent_akka_ActorCellMonitoringAspect
    at org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect.aspectOf(ActorCellMonitoringAspect.aj:1)
    at akka.event.EventStream.publish(EventStream.scala:26)
    at akka.event.LoggingBus$class.startStdoutLogger(Logging.scala:88)
    at akka.event.EventStream.startStdoutLogger(EventStream.scala:26)
    at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:537)
    at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
    at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
    at org.eigengo.monitor.output.statsd.AkkaIOStatsdCounterInterface.<init>(AkkaIOStatsdCounterInterface.scala:28)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
    at java.lang.Class.newInstance0(Class.java:372)
    at java.lang.Class.newInstance(Class.java:325)
    at org.eigengo.monitor.agent.akka.AbstractMonitoringAspect.createCounterInterface(AbstractMonitoringAspect.aj:30)
    at org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect.<init>(ActorCellMonitoringAspect.aj:46)
    at org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect.ajc$postClinit(ActorCellMonitoringAspect.aj:1)
    at org.eigengo.monitor.agent.akka.ActorCellMonitoringAspect.<clinit>(ActorCellMonitoringAspect.aj:32)
    ... 18 more

[janm399]

I know about it, but that's all I can tell you. It has something to do with the configuration of the dispatchers / threading behaviour of the mailboxes. Essentially, it appears as though the DefaultLogger does not reply to InitializeLogger on line 200 of the LoggingBus.

    import akka.pattern.ask
    // Bantha poodoo here
    val response = try Await.result(actor ? InitializeLogger(this), timeout.duration) catch {
      case _: TimeoutException ⇒
        publish(Warning(logName, this.getClass, "Logger " + name + " did not respond within " + timeout + " to InitializeLogger(bus)"))
        "[TIMEOUT]"
    }

[vonnagy]

Jan, the error that you were seeing with the LoggingBus is not that same that I was seeing. The error associated with this issue is different. the pull request #90 addresses this issue, but not the LoggingBus one. The error you mention still occurs when setting includeSystemAgents: true.

I will enter a new issue for the LoggingBus error.