rbair23
commented
1 year ago Thanks @jasperpotts for filing this. The noise was so bad I disabled INFO level logging from the platform in the services tests.
Our logging policy is roughly to log at the following levels for the following types of things:
- FATAL: If the error will cause System.exit
- ERROR: If an error occurred in the system that will cause misbehavior and likely render the node inoperable.
- WARN: If something a little dodgy happened, but we can probably recover fine. For example, perhaps the node is starting without TLS because the certificate was missing or bad.
- INFO: For general information useful to devops, or customers hosting their own nodes. For example, general information on startup, or printing when the node made a new connection with another node, or when a new address book was made effective, or things of that nature. These are the kinds of things somebody operating a node will want to know.
These four levels are the ONLY levels that are logged by default in production.
Critically, we do not log errors to user input. For example, if a user gave us a bad transaction, we don't log that. We can track it in metrics, but we don't log it. If a node fails due diligence, we don't log it. The principle here is that a dishonest or malicious user or node must not be able to trigger a log storm on the node.
- DEBUG: Developer-centric logging information not covered above
- TRACE: Developer-centric and extremely noisy log information not covered above
Problem
When looking for exceptions in database logs they are very hard to fine because of the crazy amount of noise like:
2023-08-11 09:36:14.719 536528 INFO JASPER_DB <<jasper-db: Merging #0>> VirtualDataSourceJasperDB: [FileService.FILES] Starting Small Merge 2023-08-11 09:36:14.719 536529 INFO JASPER_DB <<jasper-db: Merging #0>> MemoryIndexDiskKeyValueStore: [FileService.FILES_internalhashes] No need to merge as 0 is less than the minimum 8 files to merge. 2023-08-11 09:36:14.719 536530 INFO JASPER_DB <<jasper-db: Merging #0>> HalfDiskHashMap: [FileService.FILES_objectkeytopath] No need to merge as 0 is less than the minimum 8 files to merge. 2023-08-11 09:36:14.719 536531 INFO JASPER_DB <<jasper-db: Merging #0>> MemoryIndexDiskKeyValueStore: [FileService.FILES_pathtohashkeyvalue] No need to merge as 0 is less than the minimum 8 files to merge.Solution
At least for do nothing merges I don't think logging is helpful. Maybe even logging at all for small merges might not make sense any more. Or just move them from info to trace or debug levels.
Alternatives
No response