Operation logs improvement

[6367f766]

Is your feature request related to a problem? Please describe.

As a device owner I would like to see the logs of operations, and to query them from Cumulocity UI.

As pointed out by @mstoffel-sag, it is hard to debug operations called by the mapper. This occurs because stdout and stderr are ignored.

Describe the solution you'd like

• operations stdout and stderr is written to /var/log/tedge/agent
• operation logs can be queried from Cumulocity UI (this should be the case if file is created in /var/log/tedge/agent and follows the same naming pattern as software operations.

Additional things to consider during grooming

• How many logs do we keep for each operation? See this.
• Moving operation_logs (the code that creates the log files) out of agent, into a crate and reusing the code in the mapper.
• Who can read /var/log/tedge/agent/ directory? execute_operation method is inside the mapper code.

[albinsuresh]

It'd be better to explicitly state that this story is only about collecting the logs locally on the device at /var/log/tedge/agent/c8y_Operation-xxx.log and NOT about making them retrievable from the connected cloud as well. But, that feature should be a natural extension to this story to make the debugging more seamless from the cloud.

Collecting the logs of cloud specific operation plugins to cloud-specific log locations could also be considered for consistency. So, the log of c8y_ConfigurationDownload operation should go to /var/log/tedge/agent/c8y directory and not to /var/log/tedge/agent directory directly.

[makr11st]

I'd prefer to have the operation execution moved to the Agent before the change is made. This would allow for utilisation of the LoggedCommand already in the Agent and revert the mapper converter to non async state.

[rina23q]

We will do

• operations stdout and stderr is written to /var/log/tedge/agent
• How many logs do we keep for each operation? See this. -> 5 logs
• Moving operation_logs (the code that creates the log files) out of agent, into a crate and reusing the code in the mapper.
• Who can read /var/log/tedge/agent/ directory? execute_operation method is inside the mapper code. -> All users can read/write it.

[6367f766]

Hi since grooming #1077, I've had some design questions regarding this ticket.

What the problem is

• 1077 assumes filtering by log type is possible. This is currently NOT implemented.

  • for this, pull request #1096 has been created.

The issue is that I see this fix as a temporary solution only. If it is a solution at all!

Questions that require clarification

To do it properly (once and for all) we need to decide where to store the logs. The question here is:

• Do we store the logs in the same folder as previously existing logs? /var/log/tedge/agent? Note that the question of renaming or getting rid of the agent folder in this path entirely is a different discussion

I am going to assume the answer to this questions is yes. But I think the issue stands regardless.

Here is where the issue arises. Inside /var/log/tedge/agent we can see the software-management logs, made up of software update and list:

const UPDATE_PREFIX: &str = "software-update";
const LIST_PREFIX: &str = "software-list";

and the files created are the following:

software-update-(timestamp).log
software-list-(timestamp).log

These two file names, are grouped as "software-management", so this raises the question:

What do we group log operations as? Do we even want to group them by type? If users are implementing their own operation do they want to always retrieve logs for other "c8y-operations"?

My current approach has been to make the file prefix equal to the operation name. Namely, if we asked for a c8y_LogfileRequest operation, then a c8y_LogfileRequest-(timestamp).log would be created in /var/log/tedge/agent. This then allows a user/device owner to query only c8y_LogfileRequest logs from c8y UI.

But these are now two different conventions that require different logic.

I need some further clarification here to know how to proceed. Thank you

[didier-wenzek]

The fix for this issue should focus on producing a log file for each operation request - no more no less. Publishing these files on Cumulocity will be addressed by https://github.com/thin-edge/thin-edge.io/issues/1077. Meantime, https://github.com/thin-edge/thin-edge.io/pull/1096 is a quick and simple fix to have these file published.

[albinsuresh]

My suggestion would be to create logs under dedicated directories for each "logical operation" like /var/log/tedge/agent/software-management, /var/log/tedge/agent/c8y_LogRequest, /var/log/tedge/agent/c8y_CustomOperation etc. Having such dedicated directories will enable us to group multiple "c8y operations" like software list/update into a single software-management directory as well.

While implementing #1077 these config paths will be configured in the c8y_log_management_config.toml as follows:

files = [
    { path = '/var/log/tedge/agent/software-management/*',  type = "software-management" },
    { path = '/var/log/tedge/agent/c8y_LogRequest/*', type = "c8y_LogRequest" },
    { path = '/var/log/tedge/agent/c8y_CustomOperation/*', type = "c8y_CustomOperation" }
]