Hiding quotas and watermarks when unused in fmt_json

[sj-aws]

For continuing discussions from https://github.com/awslabs/damo/issues/104

[sjp38]

@honggyukim , I'm waiting for answer to below question. No rush. Just reminding and making the status clear.

Thank you for the clarification. And I agree those can be hidden, and that will make the output short. But, I'd like to further understand what is the problem of the current long output, and why.

For example, do you need to read and modify the output, and doing so with long output is painful? Why? Due to absence of existing tools that optimized for that? Due to your special workflow? What tool you use, what kind of modifications and reading you do for what?

We can definitely make the change, but it will impose some complexities in my opinion. I'd like clearly understand if the benefit of the change really outweighs the increase of complexity.

(https://github.com/awslabs/damo/issues/104#issuecomment-2278362029)

[honggyukim]

We can definitely make the change, but it will impose some complexities in my opinion. I'd like clearly understand if the benefit of the change really outweighs the increase of complexity.

I think the current unused quotas and watermarks info rather increase complexity for general users. Why not hiding them to make the json or yaml output compact and concise?

[sjp38]

Why not hiding them to make the json or yaml output compact and concise?

Sorry for late response.

I'm not saying we shouldn't hide those. I'm trying to better understand the estimated amount of the benefits / complexities.

For more contexts, let me explain why I'm not showing the benefit big enough on my own, but asking you to enlighten me. For DAMON-control damo commands (e.g., start and tune), we provide two types of command line options. The first one can be called "partial DAMON parameters specification options", while the other one can be called as "full DAMON parameters specification options". We have the two types of options for different users.

The "partial DAMON parameters" specification options are for users who want to utilize DAMON in simple ways, and therefore want to minimize the command line. E.g., damo start --damos_action pageout --damos_access_rate 0% 0% --damos_age 0s 2m. In ideal world, this should be enough.

However, for more creative and complex use cases, long length of commands are inevitable. Such long commands are difficult to not only type, but also missing or mistyping some of options. We therefore introduced the new type of options for "full DAMON parameters specification". The option should support full DAMON parameters, while easy to use. For that, we had to define a decent format of the specification data, and a way to conveniently maintain/modify the data. It seemed not a simple work, so instead of making our owns, we decided to use json format. The major intention was to offload the easy maintenance and modification to other tools that users can select. That is, because the json format supports files, users can maintain it as a file using their favorite file systems. Because there are many tools for editing json data, users can edit it using any other decent json editor tools. And therefore the purpose of the json support from damo, namely fmt_json command, was more focused on simply letting users don't forget option names and type of values. Hence fmt_json exports full options. The assumption is that the users can use their own json editor that could simplify the json content.

Given the assumptions under DAMON full parameters option, the benefit of making it simpler is unclear to me. In a way, it even seems it might make it worse since the simplified output might make users missing adding some options, or mis-type those. I'm actually rather concerned about some DAMON parameters that currently omitted from the full parameters json output by default. For example, if no DAMOS scheme is given, the output simply gives an empty list for the schemes field.

Of course, the assumption might be wrong. For example, users might not want to use other json editor for some reasons. I want to better understand which part of my assumption is wrong, or I'm missing what important things that users need.

[sjp38]

Now I'm thinking about adding features for damo args damon output modifications (https://github.com/damonitor/damo/issues/3#issuecomment-2466877494). As a part of it, we would want to give user simplified view of the currently editing kdamonds, like damo report damon output format. I think this can keep the json or yaml files hold entire parameter while give users simple understanding of current parameters. E.g.,

# damo args damon > foo.json
# damo args damon --read foo.json
kdamond 0
    context 0
        ops: paddr
        target 0
            pid: 98953
            region [4,294,967,296, 68,577,918,975) (59.868 GiB)
        intervals: sample 5 ms, aggr 200 ms, update 1 s
        nr_regions: [10, 1,000]

Using this, users who have their preferred json or yaml viewer/editor can keep using the damo args damon output. Meanwhile, users who don't have such their own tools can use damo. Could this be a solution for your use case, @honggyukim ?

[sjp38]

@honggyukim , I didn't get your answer to the above question, but I think the feature for human-freiendly read of DAMON parameters could be useful in general. So just implemented and pushed it as another format of damo args damon. The documentation is available at https://github.com/damonitor/damo/blob/a4120f90911abb6e3ef997191d6b94528afe1a9e/USAGE.md#report-format. It is only for reading, not for writing, though. That is, you cannot directly edit the report format output and feed it to damo. Please let me know if it solves your issue.