eigengo / monitor

Library for monitoring the Typesafe stack-based applications.
Apache License 2.0
127 stars 24 forks source link

Injecting a monitoring actor #79

Open janm399 opened 11 years ago

janm399 commented 11 years ago

It is possible to create the monitoring actor in an existing actor system. Let's explore that. It would mean advising a good pointcut in the ActorSystem, and creating the appropriate monitoring actor in the advice. Conceptually, we're talking about

aspect Pointcuts {
  // pointcut to a method that executes *just after* the ActorSystem has been initialized
  pointcut actorSystemCreated(ActorSystem actorSystem) : 
    execution(/* magic   mushrooms here */) && target(actorSystem);
}

aspect Agent {
  // it is possible that one could monitor multiple actor systems in a single application,
  // so we need to "know" which ActorRef to send the message to
  private Map<ActorSystem, ActorRef> monitoringActors = new ConcurrentHashMap<>();

  // instrument the AS creation to "inject" our monitoring actor.
  after(ActorSystem as) : Pointcuts.actorSystemCreated(as) {
    ActorRef ma = as.actorOf(Props(new OurOwnMonitoringActor(...));
    this.monitoringActors.put(as, ma);
  }

}

And then, in the advices that actually produce the monitoring information, look up the monitor ActorRef for the given ActorSystem, and send the monitoring messages to it. This allows you to "inject" a monitoring actor into an already running actor system, without having to modify its source. It is also the one thing that makes me uneasy. What if I wanted to tune the dispatcher for the monitoring actor; what if I wanted to keep the monitoring ActorSystem completely separate from the ActorSystem being monitored? That would be difficult with this approach. However, the current solution does just that, so it's actually a moot point now.

Unfortunately, the AkkaIOStatsd... implementation of the CounterInterface suffers from mailbox threading problems. If you try using it in the example application, it will fail with "failed to initialize the logger..." message. It appears as though the InitializeLogger does not reach the instance of the logging actor. The solution above would most likely fix that, because we would be creating an actor in an already running ActorSystem.

I'll dig into this further; I think it may have legs.