add pattern - role:transport,type:balance,get:stats

rjrodger commented 8 years ago

the balance-client should collect performance stats on the health of targets, this can then be used to implement circuit breakers etc

cjihrig commented 8 years ago

Is this implemented anywhere else that I can use as a reference?

mcollina commented 8 years ago

@cjihrig regarding health, I don't think so. Mainly because we need to define what is health for a target.

I would say that the only metric that is available is "number of messages sent but not replied to" a given target. Not sure how to get it in a non obtrusive way (and without a perf hit).

rjrodger commented 8 years ago

As a quick win, you can repurpose the rolling-stats module as used in Seneca main and use it in the same way to count and time messages responses: https://github.com/rjrodger/rolling-stats https://github.com/senecajs/seneca/blob/master/seneca.js#L1068 /cc @geek

What you will need to figure out is how to handle:

primary pin may be a glob, so you need to capture on args.meta$.pattern to get the concrete pattern
different target lifetimes - a target at a particular location (ip and port) is actually a series of temporal incarnations (restarts, updated deployments, change of underlying AWS instance, etc, etc), so identity cannot just be expressed by location

/cc @mcdonnelldean it should be possible to capture and chart this data into the dashboard - consider how you would want to do that

@cjihrig the purpose of the get:stats pattern is to provide information to balancing strategies - e.g reduce load on slow service etc

rjrodger commented 8 years ago

@cjihrig it would help you a lot to run through the nodezoo workshop: https://github.com/rjrodger/nodezoo

in particular, you can add balancing scenarios:

have multiple search services and use the balance-client over them, actor model
have multiple npm and git services, publish model (who cares if they perform the same work twice! redundancy ftw :) )

mcdonnelldean commented 8 years ago

@cjihrig We should probably talk in the new year on this too. I have some examples of charting similar info I want to show you for context.

mcdonnelldean commented 8 years ago

@rjrodger I might close and reframe this. There is a lot of could ideas in this. I have someone in mind but I think there are a lot of details we could pull out of this. I would be super interested to know the round robin stats for a particular service for instance.

For now I've added enhancement but this is something I definitely want to explore as I don't think we take nearly as much advantage of these sorts of patterns as we can and should.

dgonzalez commented 8 years ago

Hi @mcdonnelldean. I am working in something related to stats and circuit breaking and I am actually modifying now seneca-balance-client (I have done it but I am trying to debug a tricky error) to introduce circuit breaking capabilities as well as stats collection. I think it might be worth having a talk to brainstorm ideas.

mcdonnelldean commented 8 years ago

For sure, I am out all this week but let's catch up when I get back.

dgonzalez commented 8 years ago

Perfecto. I'll put something down in the diary.

mcdonnelldean commented 8 years ago

Do, send me on a calendar invite. @mcollina yourself or Dave may want to sit in on this each to see what sort of stats are being emitted?

dgonzalez commented 8 years ago

Yup. Id like to sit in there. I have a branch already with something going on. Ill send the invite. On 26 Apr 2016 12:12 p.m., "Dean McDonnell" notifications@github.com wrote:

Do, send me on a calendar invite. @mcollina https://github.com/mcollina yourself or Dave may want to sit in on this each to see what sort of stats are being emitted?

— You are receiving this because you commented. Reply to this email directly or view it on GitHub https://github.com/rjrodger/seneca-balance-client/issues/7#issuecomment-214705438

senecajs / seneca-balance-client

add pattern - role:transport,type:balance,get:stats #7