Add a statistics page for different measuring points from Bifrost into Mimir

[einari]

Look at something like this for gauges : http://www.justgage.com

[markwoodhall]

Hi,

I am looking at giving this issue a go and wanted to try and promote some discussion around what kind or metrics we would like to display?

Any input would be appreciated?

Cheers Mark

[smithmx]

Einar may have some specific reports in mind but I am wondering if it should be a bit more generic than that.

What I am thinking of is a kind of plug-in model that allows us to write a statistical report as a standalone class and then have that report be discovered and displayed in Mimir. Sure, we can implement some standard ones for things like commands that are not authorized, failed logins, etc. but we also leave it extensible for domain specific statistics.

We've got a pattern that we tend to follow in Bifrost. When we identify a concept, we create an interface that represents this concept. If it has behaviour, we put methods on the interface. If not, we leave it as a marker interface. At startup, we discover all the instances of the interface so that the developer doesn't need to remember to hook things up. See something like Validation or Security for an example of how this works.

This is probably more complicated in that we need two parts to it. We need the code that is going to generate the statistics and then we need to display it. The server side stuff is probably straightforward in that it would be something like

interface IGenerateStatistics { dynamic Generate() }

but we'd need something that was able to bind to the result and display it. Probably a default one that just lists key value pairs but with the possibility of writing a custom View / Display / Binding / whatever it is! Front end part is not my strong suit.

Then again, I could be completely off track with what this whole thing! :-)

[einari]

Michaels is spot on IMO. It needs to be pluggable. I would suggest introducing something that the command coordinator can call for commands and something the event processor can call for events. These subsystems would then use the typically the ITypeDiscoverer for discovering types these sub systems can use to generate the statistical data.

So something like:

interface ICommandStatistics { void WasHandled(ICommand command); void HadException(ICommand command); void HadValidationError(ICommand command); void DidNotPassSecurity(ICommand command); }

And then the plugins would have the same methods, but return a statistics object with metadata that we can store. The ICommandStatistics would then be in charge of storing this.

interface IGenerateStatisticsForCommands { }

For Events that would be something like:

interface IEventsStatistics { void WasApplied(IEvent event); }

And when I started looking at what I had wrote thus far, EventSubscribers are something of its again, so a third thing for now that should have statistics:

interface IEventSubscriberStatistics { void WasProcessed(IEventSubscriber subscriber, IEvent event); void HadException(IEventSubscriber subscrbier, IEvent event); }

Then we would need something that persists this, and configuration to go with it. Introduce a namespace called Statistics, don't stick any of the above in that - they should be close to the concepts they represent. But in the statistics namespace you can put in IStatisticsRepository or something with a better name . And we would need to implement that for RavenDB, MongoDB and NHibernate for now. We are in the midst of trying to phase out the usage of IEntityContext that we have in entities, it causes headaches during configuration. But right alongside with the IStatisticsRepository you put typically a NullStatisticsRepository that will be used by default.

Take a look at the configuration and how things are put together, we have an interface representing the different things that you can configure. So it would be natural to introduce a IStatisticsConfiguration and then have ways of specifying with extension methods fluently the storage - so for Raven, Mongo and NHibernate you would have extension methods for them..

[einari]

A little follow up to the statistics object that the plugins would return. What I think I'm trying to say is that it needs to return data if it impacted statistics. The "mother" system - being the ICommandStatistics or others would then just take that and combine it with existing data. The object coming out would have something that categorizes the data, so in theory for now at least, its basically a string that needs to be returned almost. From the category you would look up something that matches the name of plugin and the category returned. If the plugin decides that the object coming in doesn't impact its "categories" it represent, then it should return some kind of null representation and the "mother" should then just exclude it.

What I'm getting at is that the ICommandStatistics should be in charge of the actual aggregation and the plugin just be a visitor in the pipeline of statistics.

[markwoodhall]

Thanks for the information, gives me plenty to work with.

[markwoodhall]

I'm trying to understand at the moment what we actually persist, for example if we pass in a command to the WasHandled method on ICommandStatistics and that is responsible for delegating to a repository implementation to persist, what data do we pass it?

Do we just persist a simple statistic object that has some kind of category (i.e 'Handled') ? I presume we don't want to persist the actual command.

[markwoodhall]

Just noticed I seem to have line endings configured differently, I'll fix that...

[einari]

I noticed.. Kinda hard to figure out what was going on there.. :)

But for your question. I'm not 100% sure what it is we're exactly looking for, but I imagined the methods ICommandStatistics representing the different steps, being top level categories - then you'd have the plugins returning information telling the sub category, if any, the command was recognised as.

So for instance, a Command coming in could be matched for a BoundedContext - which could be one plugin, another plugin could match it for Module and return that it belongs to a Module so and so.

The Plugins are just visitors in the pipeline reporting back wether or not it was affected. So maybe the plugins shouldn't really returning anything, but rather the Statistics object that the ICommandStatistics is responsible for persisting, and it could have methods on it for reporting if a match was found somehow by the plugin.

Since we haven't formalized BoundedContext and such yet, I suggest you create a plugin for now that deals with Namespace - so we can see how many commands gets handled, failed, invalid, not authorized and such per namespace.

Once this is in place, we can start becoming creative and really get to some cool things!

[markwoodhall]

So, since I'm progressing a little now I thought I'd get some thoughts of you guys. At the moment I've hooked into the CommandCoordinator and am generating statistics that look like this...

    {
      "Categories": [
        {
          "Key": "CommandStatistics",
          "Value": "WasHandled"
        }
      ]
    }

Obviously we can have entries for WasHandled, HadValidationError, HadException or DidNotPassSecurity and plugins can inject data via a discovery process. For example, in Bifrost.QuickStart say I have an EmployeeStatistics plugin that looks like this....

public class EmployeeStatistics : IStatisticsPlugin
{
    ICollection<string> _categories = new List<string>();

    public string Context
    {
        get { return "EmployeeStatistics"; }
    }

    public ICollection<string> Categories
    {
        get { return _categories; }
    }

    public bool WasHandled(Commands.ICommand command)
    {
        if (command.GetType() == typeof(RegisterEmployee))
        {
            var register = (RegisterEmployee)command;

            Categories.Add("RegisteredEmployee");

            if (string.IsNullOrEmpty(register.SocialSecurityNumber))
                Categories.Add("RegisterEmployeeWithNoSocialSecurityNumber");

            return true; 
        }

        return false;
    }

    public bool HadException(Commands.ICommand command)
    {
        return false;
    }

    public bool HadValidationError(Commands.ICommand command)
    {
        return false;
    }

    public bool DidNotPassSecurity(Commands.ICommand command)
    {
        return false;
    }
}

Then I get a statistics object that looks likes this...

    {
      "Categories": [
        {
          "Key": "CommandStatistics",
          "Value": "WasHandled"
        },
        {
          "Key": "EmployeeStatistics",
          "Value": "RegisteredEmployee"
        },
        {
          "Key": "EmployeeStatistics",
          "Value": "RegisterEmployeeWithNoSocialSecurityNumber"
        }
      ]
    }

Is this looking like the kind of thing we wanted, any thoughts or guidance would be appreciated. :+1:

Also, if this is the kind of object we are persisting whats the best way to aggregate them up into counts for a dashboard style view?

[einari]

Hi, looking good. You're getting forward.

We've had a couple of discussions, and we want to introduce something that allows us to do cross-cutting concerns without being obtrusive. This is a very good example of it.

What we've discussed is a way to hook up a convention for the container that discovers aspect describers, and what we could effectively do is to generate a proxy for whatever CommandCoordinator was configured and basically do statistics based on the end-result.

So what I suggest you do is to change the ICommandStatistics interface to basically only have one method - Handle() or similar that takes the CommandResult and then generates the statistics based on that.

For now, the most important statistics we can have is the count of things. So how many handled methods within a BoundedContext, Module, Namespace and so forth, how many with errors and so on. So I suggest the plugins to be just visitors in the pipeline. Instead of them returning strings in their implementations with categories and such, they can be called by the ICommandStatistics as visitors to a Statistics object that they can report their categories on. This way they can extract information from the command and use the information as a category. Take namespace or command name for instance, this could just be used directly. The context is also known in the sense that the type of the plugin is the context.

As for the IStatisticsPlugin name, I'd rather have it related back to the Commands and say ICanReportStatisticsForCommand or similar. We tend to have different naming of things and we use the I not as a prefix, but try to make it read out directly

[markwoodhall]

Hi,

OK, sure, I'll have a look at making those improvements. This was a bit of a unknown for me and I now think I have a bit more clarity so this should help. :)

This is some good information, thanks! :)

[einari]

Great.
I know we do things in a bit particular way - so hope you don't feel discouraged!

[einari]

About the cross cutting thing, btw. What this means, for now, just stick everything in one method and deal with the CommandResult. It has enough information to tell the different measuring points we want to measure. If its successful; success, exception, errors, validation errors, security.. So forth.. All is in there.

Btw: Love your fixup in the CommandResult!! So much clearer when you stick things in methods that tell what it is doing instead of "blah != null"..

[markwoodhall]

Dont feel discouraged at all, feel free to comment as you find :)

[einari]

Good. Then you won't mind me changing my mind on the Handle for Statistics, but rather it be Record() instead. :)

Btw: have a look at #215 - this is the aspect part.

[markwoodhall]

I've tried to take on board some of your suggestions, improved the naming of some items and tried to better manage setting context on a statistic.

I've done this by introducing an interface representing a 'visitable statistic' and only exposing this to statistics plugins. This doesn't expose the ability to set context. The context is inferred when the plugin is discovered.

Hopefully, you think this is an improvement but again, feel free to give any feedback.

[einari]

Looking good. I think I'd like to get a pull request for the backend code and look more at it in context of code and usage. Its a bit hard right now to see the context.

[markwoodhall]

Ok, sure, I will do that at some point today then. I've not done anything with events yet, just commands. :)

[einari]

No probs. Great to the commands in a good shape, then we have the pattern for the rest that we want to measure.

[markwoodhall]

This is what the stats page in Mimir looks like at the moment, its still a work in progress..

[einari]

Wow..
That looked better than I imagined in my head :)

Great work..!! Can't wait to get it all fit together and some SignalR love sprinkled on top.

Einar Ingebrigtsen Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Saturday 9. March 2013 at 22:40, Mark Woodhall wrote:

This is what the stats page in Mimir looks like at the moment, its still a work in progress..

— Reply to this email directly or view it on GitHub (https://github.com/dolittle/Bifrost/issues/147#issuecomment-14671274).

[markwoodhall]

Yeah, it works quite nicely.

I think longer term there needs to be some thought around context filtering and selection. I.e. to view core statistics or statistic from a particular plugin/context but its getting there.