search

waku-org / nwaku

Waku node and protocol.
Other
200 stars 52 forks source link

Add logging of hashes to all nodes #2474

Closed chair28980 closed 5 months ago

chair28980 commented 7 months ago

For each node in the fleet, we want to log at INFO level any message that comes in and out.

This should be logged for any protocol:

1) Relay, when a message comes in, ideally with the peer ID of the sender 2) Relay, when a messages goes out, ideally with the peer ID of the receiver 3) Store, when a message comes in and when a message is stored 4) Lightpush, when a message is received from a lightpush client, with peer ID 5) Filter, when a message is pushed to a client, with the peerID

Some of these might already be logged, but we would like to have them enabled for any node at INFO level, instead of only a selection of nodes.

Logs should be available in kibana.

I think that should cover all the cases, but let me know if there's more to be logged.

Ivansete-status commented 7 months ago

PR with the suggested log changes: https://github.com/waku-org/nwaku/pull/2495

An image with those changes was deployed in status.test-amsterdam node, on two occasions: the 4th and 7th of March. On the 7th of March, I redeployed the same image again because the status.test-ams node got automatically updated due to a Jenkins job, which I disabled ever since.

I sent some messages in our "logging" Discord channel to inform about that.

For further reference, the logs can be checked with:

https://kibana.infra.status.im/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-60m,to:now))&_a=(columns:!(logsource,message,kv_raw),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:ffcc22b0-0116-11ed-9719-cdd3b483481c,key:data_center,negate:!f,params:(query:do-ams3),type:phrase),query:(match_phrase:(data_center:do-ams3))),('$state':(store:appState),meta:(alias:!n,disabled:!f,index:ffcc22b0-0116-11ed-9719-cdd3b483481c,key:kv_raw,negate:!f,params:(query:'*msg_hash*'),type:phrase),query:(match_phrase:(kv_raw:'*msg_hash*')))),hideChart:!f,index:ffcc22b0-0116-11ed-9719-cdd3b483481c,interval:auto,query:(language:kuery,query:'fleet:%20%22status.test%22'),sort:!(!('@timestamp',desc)))

cc @cammellos, @john-44

Ivansete-status commented 5 months ago

I close this because it has been covered by: https://github.com/waku-org/nwaku/pull/2640

and by: https://github.com/waku-org/nwaku/pull/2495 ( this one has libp2p changes that hasn't been added to nwaku-master )