Move all user-friendly reporting from sonalyze into naicreport?

[lars-t-hansen]

This is a little speculative but worth discussing, by and by.

More and more, sonalyze is becoming a query component against sonar data, used more for its query capabilities and its csv and json outputs than for direct reports to people. The formatting of reports, both human-readable and not, is increasingly the domain of naicreport. It may be that more formatted / cooked reporting should move into naicreport, and that there should be a custom protocol between sonalyze and naicreport to facilitate the transfer of raw query data. Presumably both languages have libraries for protobuf or similar fast binary encodings.

(Another alternative is to move more things or all things into sonalyze but I'm not fond of using Rust for all this high-level work, it's not well suited for that. And sonalyze remains in rust because sonarlog is in rust, and that remains in rust because of jobgraph.)

[lars-t-hansen]

This is becoming a little more urgent as the reports are becoming more sophisticated. The flat text csv output - or indeed the json array-of-objects - is no longer a suitable conduit for data between the components. Consider load reporting with clustering. In this case, sonalyze computes a flat profile for the merged host but it also computes an aggregate system configuration structure for the merged host based on the host names that go into the cluster. When naicreport invokes sonalyze, and it receives the flat profile, it must itself recompute the system configuration structure, based on host names extracted from the report records received. And both naicreport and sonalyze must agree on what a particular set of host name patterns means (see also #247). A better solution would be for naicreport to receive all these data from sonalyze in a structured form, typically as a nested json structure for now, that naicreport then formats for display as it needs to.

[lars-t-hansen]

In #266, the communication between sonalyze load and naicreport load was changed to use a richer JSON format that contains system information in addition to data. This is possibly the direction we want to go in, unless we make sonalyze into a very low-level thing that receives system information and pumps out tables of data for further processing by naicreport.

[lars-t-hansen]

I think this is "not actionable" any longer, the division of responsibilities is working out OK. By and by, sonalyze will be rewritten in Go and we'll have a better toolbox for producing reports from within sonalyze, too.