Open KillerInstinct opened 9 years ago
Agreed on most points. @rep already proposed the task aware logging a while ago but we haven't put it to production yet.
In addition to the points listed here I'd also like to see a per-task error.log
which contains a dump of all exceptions that happened for this particular task. Naturally each task should have no exceptions at all, but it has happened various times where incorrect usage of 3rd party libraries caused exceptions, e.g., dpkt, pefile, volatility, etc.
Btw, in the case of rolling logs, we do have to make sure that logs are not magically deleted. E.g., I still may want to see the logs of an analysis half a million analyses later ;)
Stumbled back across then when I was checking Issues I was involved in. I ended up coding the third point because an upstart job fails to account for when you want to reprocess and analysis, you're still only left with logging to STDOUT. This solves that.
Relevant commits: https://github.com/spender-sandbox/cuckoo-modified/commit/e22b9adf2144c5617738096fe879942277e531bd https://github.com/spender-sandbox/cuckoo-modified/commit/00f176cd61f1c6891e0c8550a422e02a7c72c03e
Feel free to use/improve.
Having forgotten about this issue, some of the suggestions have been implemented by now. In particular: more task ID logging, per-task logging, process.py logging (in #863), and the task aware logging. I guess the rest have yet to be done :-)
There are multiple enhancements that could be done to the current logging architecture.
Add a log handler to process.py. Currently there is only STDOUT. If you want to reprocess an analysis, there are no retainable logs unless you pipe it to a file and after reprocessing, the logs for that task in cuckoo.log may not be applicable anymore. For people using process.py in auto mode, there is no logging at all unless you force it with something like an upstart job, or redirecting stdout to a file, etc. Would be wonderful to have this be logged to its own file.Task aware logging. All logs specific to a particular task should be monitored in a task-by-task basis so that we can dump them into MongoDB. We could then download/display them in the Admin tab (or similar). This would mostly eliminate the need to log into the server to get logs, and would make Cuckoo be more 'appliance' like.