Magickbase / infra

0 stars 0 forks source link

Set up Notification for community public nodes, CKB explorer APIs, and Godwoken explorer APIs #36

Open Keith-CY opened 1 year ago

Keith-CY commented 1 year ago

As a public service provider, the status of services should be fully monitored and synchronized to users timely.

Now we have pretty much monitors of services, as

It's time to push notifications to our customers.

For me, the simplest solution is to enable a subscription of service status so messages can be pushed to the subscribers.

But what message and how often should the message be pushed should be considered.

Any suggestion about this feature? @mdzh521 @ShiningRay @zmcNotafraid @iamyates @zhengjianhui @Danie0918 @Sven-TBD

Danie0918 commented 1 year ago

Service monitoring is divided into daily reports, alarms and announcements

The daily record records the daily monitoring data of yesterday's UTC 0:00-23:59 or UTC 8:00-7:59 time period, sent at UTC 0:00 or 8:00 every day, including the overall status service (chart), abnormal times and Time period, upgrade, etc., and can support custom subscription content in the future.

When an exception occurs, an alarm needs to be issued, and criteria for judging the exception need to be defined, such as rollback, shutdown for a certain period of time, or other major failures. Notify users of important situations while reducing user harassment.

Announcements are usually sent before the planned operation and maintenance business, edited by relevant personnel and pushed to users, such as business deployment and upgrade.

mdzh521 commented 1 year ago

Added a simple version, the follow-up in the continued optimization, the current implementation of the function is when the block is not updated for five minutes will be alerted. The effect is shown in the figure:

Image

https://grafana.layerview.io/d/tSqBCSE4k/ckb-node?orgId=1&refresh=5s