Open mDuo13 opened 5 years ago
It might be nice for the log to indicate "happy" once a set of requirements are met, e.g.
peers
>= 8server_state
: proposing
or full
validated_ledger.age
is undefined or is less than 6After starting a server, you usually want to wait until these requirements are met before relaxing and moving on to other tasks. If the log said something once these were met, it could save the effort of polling server_info
and digging through irrelevant fields to find the few things that really matter.
IMHO, the server should also log a message when it loses sync (or becomes unhappy re @intelliot's comment). My reasoning is that if a user is using the log to diagnose, then the last message on the topic they may see will be "Happy happy", even if it's months old, and the server hasn't had peers (and thus is unsynced) in a week.
This is not something that should be a "warning" imho - and if you are monitoring a node by looking at log messages instead of the output of server_info, you're going to have a bad time anyways.
I guess we could also just make a health check method. But I kind of think a log message saying it's OK is still sensible.
Slightly different idea but, I think a different method that summarizes the key health metrics and maybe has one metric that is a function of the others (per Elliot's post) is a good idea. This avoids the digging through server_info and is mentally more useful. I do not like the idea of dumbing it down to one single "green/yellow/red" status, because we should push operators to understand the key metrics they need to monitor and how they affect their servers. Something like "opt/../rippled server_health" which then provides the 5-6 metrics and a status would be ideal.
Edit: I see you already made a issue for this :)
What we should do instead is make it so that, by default, the server displays a short “dashboard” when running, showing server state, number of peers, ledger age, whether it’s amendment blocked or UNL blocked; just a quick “important stuff” view.
Launching it with --quiet
suppressed that. Launching it with --verbose
switches to the “console logging” mode we have today.
@nbougalis do you know of any other servers / apps / clients that display a "dashboard" when running?
Would be useful to have a look at their implementation, and perhaps model based on it.
Or maybe just UDP logging with configurations for one of the myriad log viewers ?
When you start or restart a server, it writes a lot of unhappy log messages at the
WRN
level indicating that it's not on the consensus ledger, etc. but there's no indicating that the server is successfully synced other than the lack of messages.It would be helpful if, the server wrote a log message indicating success after the first time it successfully reaches the "full" or "proposing" state following each restart. To make sure it appears in the logs by default, it should be logged at the
WRN
level. (That may sound unintuitive because it's a "warning" about something working right, but it makes sense when you realize it's indicative of the fact that the server restarted.)Having this sort of log message would make it possible to confirm just from watching the logs that the server has successfully synced after a restart, and it would also help identify how long it took to sync.