Metric support

[t11omas]

Are their any plans to introduce metrics, for example the duration of a consumer? I can write an interceptor to do this currently, but I don't thinks its an accurate value as the message has already had consumption logic performed on it, i,e desterilized at this point.

So I think either a hook in point to when the message is first received is required, or build the metrics into SMB

[zarusz]

hello,

So I was thinking about adding these at some point via the OTEL. See #149. Happy to accept a PR if you want to help.

For now one could add a generic consumer interceptor that would measure the consumer processing time and log using your favorite telemetry platform.

Also some of the underlying client libraries each transport use have their own (but yes, better if SMB abstracts it all under one unified model).

Let me know your thoughts.

[t11omas]

I have done that via the generic interceptor, but the issue is that by the time it hits the interceptor, some processing has already occurred. I.e, its already been desterilized which means that time wont be included.

I will take a look at https://github.com/zarusz/SlimMessageBus/issues/149 :)

[t11omas]

Just copying this question over to this ticket:

Is there any way to see/monitor the count of messages on the in memory bus? (might be another example of a good metric to log)

We have a process that reads a lot of data and places it onto an in memory bus to be processed. It would be nice to be able to see/monitor that in memory bus

[zarusz]

Yes, that's another good metric that should be logged.

For now you could either turn on more verbose logging and then count the lifecycle log events that are indicative of counts (not great, but possible).

Add an generic interceptor as explained above and increment the counter for your metric platform, but you have that already. For in memory by default messages are not serialized/deserialized so it wont include that overhead (which is what you're looking for).

Either way, we need to capture metrics (new feature). I belive we could use OTEL as the abstraction. Would OTEL work for you? What logging/monitoring platform do you use?

[t11omas]

We are using a Hybrid bus, so both in memory and external. I understand for the in memory bus the messages don't go via serialization, but they do for the external buses.

OTEL would work for us, but I am wondering, just from a design point of via, could it somehow be abstracted like what you have done for the transports and serialization.

For example, and don't get caught up on the names or signatures I use here, they are just used to illustrate the point.

public interface IConsumerObserver 
{
   void ConsumeStarted();

   void ConsumeError();

   void  ConsumeFinished();

}

public interface IInMemoryBusObserver
{
   void MessageAdded();

   void MessagedRemoved();

}

That way you can added in OTELMetric or extend it to support whatever platform is required

[zarusz]

Yes, that's what I was thinking too. OTEL isn't a clean abstraction (we have to drag in libraries into the SlimMessageBus.Host) in itself a clean interface would address my concern. However, OTEL has become a standard that many platforms use nowadays, and I believe users of SMB would expect to just connect it to Prometheus/Grafana/Azure App Insights and it simply working.

That way you can added in OTELMetric or extend it to support whatever platform is required

Yes, maybe the path would be SMB > SMBs abstraction (the interface above as suggested) > OTEL plugin (optional).

I will think about it. Again, please voice your needs and opinions. Feedback is much appreciated.