search

BOINC / boinc

Open-source software for volunteer computing and grid computing.
https://boinc.berkeley.edu
GNU Lesser General Public License v3.0
2.03k stars 449 forks source link

Linux journal logs: apply limit to growth #3321

Open RichardHaselgrove opened 5 years ago

RichardHaselgrove commented 5 years ago

Describe the problem On my still-young Linux test machine, the BOINC client journal (equivalent to stdoutdae.txt) has grown to 270MB in just over three months:

-- Logs begin at Tue 2019-07-02 19:42:56 BST, end at Tue 2019-10-08 12:50:44 BST. --

Describe the solution you'd like A space control mechanism similar to in cc_config.xml

AenBleidd commented 5 years ago

@RichardHaselgrove, I believe, this could help you: https://unix.stackexchange.com/a/139515

RichardHaselgrove commented 5 years ago

Thanks, but that doesn't really address the question. Looking at the journal in more detail, the user journal is empty, but the system journal is 280 MB - of which, 96% is BOINC logs. So far as I can see, the rotate and vacuum commands operate on the journal as a whole, not on definable bits of it.

As previously discussed in contributor calls, BOINC on this system (and on others like it) is a user application, not a system entity - although I've installed it as a systemd service as part of my education. I can export copies for a single unit when I want to search the BOINC logs only, but I don't seem to be able to purge them from the database while leaving other units intact.

AenBleidd commented 5 years ago

Ok, here're two things: 1 - Application must not care about journal logs rotate, vacuum and so on. This should be done either by the user or by OS based on user settings. 2 - If an application run from user account - it should write logs to user journal. Otherwise it should write logs to system journal. Both points from my prospective BOINC operates fine. I strongly not recommend move logs to user journal when running BOINC as a service (I'm pretty sure it's impossible because at the moment when service start there could be no user session present. Also I don't recommend to clean logs by BOINC itself in some way, because it's not the application domain. "Each department must stick to its own business." On Windows there are also log system like the linux one (and it's basically used mostly by services and OS itself) and any application must do nothing with it except of writing there.

RichardHaselgrove commented 5 years ago

Well, I've had a bit of a think and a refresh of #2260, which is where all this started. There's a lot of detailed discussion about the technicalities, but nothing (that I've seen so far) about who might be wanting to access these logs, and why.

The implicit assumption must be that the logs are an aid for local software maintenance - a sysadmin performing an initial install, or troubleshooting a failure. I can buy that.

But there are other use cases. My research on Sunday was to find the precise time that SETI@Home had failed on the last five Sundays - "around 03:40 PDT" wasn't really enough. So the lack of timestamps in the old-style logs would have foiled me before I'd even started: +1 for systemd. But on this machine, the 2,000 line viewable Event log last about an hour and a half: the extended history was also vital.

"Each department must stick to its own business."

I agree, but I'm not sure that BOINC's dominance of the system log is really playing by that rule. According to top, the client and the science applications are all running under user 'BOINC' - but that doesn't help, as journalctl will only output the user log for the current user, and we can't log in under the sandbox account.

So. I'd still like to explore what other alternatives are available, please.

AenBleidd commented 5 years ago

@RichardHaselgrove,

I agree, but I'm not sure that BOINC's dominance of the system log is really playing by that rule. According to top, the client and the science applications are all running under user 'BOINC' - but that doesn't help, as journalctl will only output the user log for the current user, and we can't log in under the sandbox account.

This is completely another topic. But this issue title is Linux journal logs: apply limit to growth. This is definitely must not be done by BOINC application itself.

Germano0 commented 5 years ago

I cannot reproduce the problem on Fedora.

RichardHaselgrove commented 5 years ago

May I ask where the equivalent of stdoutdae.txt - the long-term Event Log that spans BOINC restarts - would be found on Fedora?

Obviously, the volume of growth would depend on the power of the machine and the detail of optional logging chosen. Mine happens to be above average on both counts.

Germano0 commented 5 years ago

May I ask where the equivalent of stdoutdae.txt - the long-term Event Log that spans BOINC restarts - would be found on Fedora?

They go into journald logs, which maximum size is setted by journald options

RichardHaselgrove commented 5 years ago

That's my problem. With an active BOINC system, on a relatively constrained computer, my wish to keep BOINC data for later reference, but a constrained journald system log, are pulling in opposite directions: a single journald option value can't meet both objectives.

original-birdman commented 5 years ago

One thing that would reduce the boinc log size would be to find some way to prevent this entry:

Oct 16 10:21:55 benuc boinc[21636]: No protocol specified

which occurs every minute.

BryanQuigley commented 4 years ago

That's my question: is this really caused by "No protocol specified" which is issue #2256 or is it still noisy without it?