Enhance Output Handling by Piping STDOUT/STDERR to a File

[charmitro]

Description

Currently, our application captures the STDOUT/STDERR of a subprocess directly into a buffer, which is then processed and used within the application, such as for constructing an email body. This method, while effective, has its limitations in terms of scalability and efficiency, especially when dealing with large amounts of output data. To improve upon this, we propose to enhance our output handling mechanism by piping the STDOUT/STDERR of subprocesses directly to a file. This approach will not only help in managing large output more effectively but also simplify the process of logging and debugging.

Expected Benefits

• Improved Scalability: Directly writing to a file reduces memory overhead, especially for large outputs.
• Ease of Debugging: Having a physical file makes it easier to inspect the output for errors or logs.
• Performance Optimization: File I/O might offer performance benefits over in-memory operations for large data sizes.

Proposed Changes

• Modify the execve_binary function to redirect STDOUT/STDERR to a specific file instead of capturing it in a buffer.
• Implement a mechanism to generate or specify output file paths dynamically (cncDDMMYY_HHMMSS.log), ensuring no conflicts or overwrites occur.
• Adjust downstream processing (e.g., email body construction) to read from the output file if necessary.

Potential Challenges

• Managing file I/O errors and ensuring robust error handling.

[panosfol]

@charmitro What should the default path for creating the .log file be? Is home/user/ ok? And how could the user customize the path?

[charmitro]

@charmitro What should the default path for creating the .log file be? Is home/user/ ok? And how could the user customize the path?

/var/log/ or /var/log/cnc/ or user-specified.

[panosfol]

We can't create the cnc folder from our side though, should we leave that up to the user to set it up? And if we want the user to specify it, should it be a field in our config that the user specifies the path to keep the logs?

[charmitro]

We can't create the cnc folder from our side though, should we leave that up to the user to set it up?

If the directory doesn't exist, we create it

And if we want the user to specify it, should it be a field in our config that the user specifies the path to keep the logs?

Yes, a config field.

[charmitro]

@charmitro What should the default path for creating the .log file be? Is home/user/ ok? And how could the user customize the path?

/var/log/ or /var/log/cnc/ or user-specified.

@panosfol and ofcourse, $HOME/ if left unspecified and no superuser rights. So in total,

1. /var/log/cnc/
2. $HOME/ if /var/log/cnc fails due to superuser rights
3. User-specified under

   [general]
   log_location = a/path/here