Particular / ServiceControl

Backend for ServiceInsight and ServicePulse
https://docs.particular.net/servicecontrol/
Other
52 stars 47 forks source link

Implement Heartbeat indicator per Endpoint #41

Closed dannycohen closed 11 years ago

dannycohen commented 11 years ago

As Opie, when the heartbeat indicator is red, I want to drill down and see exactly which endpoints are not emitting heartbeats in a timely manner, so I can take corrective actions.

Visualization:

  1. Clicking on the heartbeat indicator displays an indicator for each logical endpoint
  2. For each endpoint:
    • endpoint should be green if sending heartbeats in a timely manner
    • endpoint should be red if sending endpoint fails to send heartbeat messages
    • endpoint name should appear below the endpoint heartbeat indicator icon
    • seconds elapsed since last heartbeat messages was received should appear next to the the endpoint name

Notes:

Demo / Acceptance tests:

Case 1:

  1. Deploy the heartbeat plugin in 3 of the 5 Video Store sample endpoints (in all except the ContentManagement and Operations endpoints)
  2. Run the Video Store sample
  3. Wait for 1 minute while heartbeat messages are sent and processed
  4. Heartbeat indicator should be green and the number 3 should be displayed
  5. Click on heartbeat indicator
  6. 3 endpoint heartbeat indicators should be displayed, all green.
  7. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator

Case 2:

  1. Kill the Sales endpoint
  2. Indicator should turn red within 1 minute
  3. Number below Heartbeat Indicator should be 2 in green and 1 in red
  4. Click on heartbeat indicator 
  5. 3 endpoint heartbeat indicators should be displayed, 2 green, 1 red
  6. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator

Case 3:

  1. Restart Sales endpoint
  2. Within 1 minute the heartbeat indicator should be green and the number 3 should be displayed
  3. Click on heartbeat indicator 
  4. 3 endpoint heartbeat indicators should be displayed, all green.
  5. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator

Case 4:

  1. Deploy the heartbeat plugin in the Operations endpoint and restart it
  2. Within 1 minute the heartbeat indicator should be green and the number 4 should be displayed
  3. Click on heartbeat indicator
  4. 4 endpoint heartbeat indicators should be displayed, all green.
  5. The name of each endpoint and the number seconds elapsed since last heartbeat messages was received noted next to each indicator
dannycohen commented 11 years ago

Moved to https://github.com/Particular/ServicePulse/issues/11