Log rotation for monero

[ChiefGyk3D]

I noticed logs were building up in my .bitmonero, and I think we need to add a log rotation to the .bitmonero folder in /etc/logrotate.d/monero this keeps 5 logs and clears it daily

/home/pinodexmr/.bitmonero
{
        rotate 5
        daily
        missingok
        notifempty
        delaycompress
        postrotate
                    invoke-rc.d rsyslog rotate > /dev/null
        endscript
}

[shermand100]

Yes, this is exactly the sort of thing I like. A tool that was there all along that I just wasn't aware of.

Currently those log files are removed every Sunday at midnight by a pinodexmr user crontab called "weekly log clean". What you're suggesting is a far better, purpose built tool. Absolutely will be implementing.

[shermand100]

@ChiefGyk Doing work on this has highlighted a couple of things as well as a bigger problem which I think needs to be patched relatively quickly.

1) Log rotation: So I played for a while using logrotate which conveniently is included with debian so seemed like a simple case of working it in to how Monero does it's logging. In the start commands for Monero the log file output location is specified to be where the 'www-data' web ui user has access to display in the 'logs' tab in the UI. This single log file (as specified by 'max-log-files=1') behaves as expected. Which means that some form of logs management must exist. But I can't figure why 100MB logs are also still generated in ~/.bitmonero So further digging brings up that by default Monero will produce a max of 50 log files of max size 100MB before rotating them. Which it seems it's trying to do in ~/.bitmonero.

Anyway long story short people have tried to work in "logrotate" but without success as detailed here https://github.com/monero-project/monero/pull/3897 It's just easier to use what Monero already has than to try and work in something else externally. Which makes sense.

So based on this I suggest start flags like --log-file=/home/pinodexmr/.bitmonero/bitmonero.log --max-log-file-size=31457280 --log-level=0 --max-log-files=2 or some variation of that gives a faster loading log file ( set 30MB here but could be reduced ) with one historical file on rotation? Then just add a hard link to this in use log file at /var/www/html/bitmonero.log so that the web server has access to it. ( symlink follow is enabled by default in the apache config which is nice)

2) However. The found issue... So I'll be honest. I just don't look at my logs. Nothing much goes wrong that wasn't because I'm working on something that I just reverse or fix, so looking into this issue I was surprised to find that the status outputs have been writing to the logs too. I mean, I'd expect to see the RPC request or validation to say that print_pl for example was called. I didn't expect to see the 1000 white nodes and 5000 grey nodes printed verbose in the log at level 0. This does explain where all this bloat is coming from since the stats call change. To clarify the previous stats method I used would have written to the log too, it's just that the old timer (crontab) for the peer list was ~4hourly so I never saw it in the log. The new rotation method cycles ~3minutes. So that's over 6000lines of just peer lists bloating logs every 3 minutes. What a stupid over-sight.

I'll move that status call back to it's own crontab on a 4 hourly run again.

Another bloaty stats call can be the tx pool. But that's more time critical?

EDIT: I'll deal with the worst offender tomorrow (the full peer list call), and could switch that back to being a 4 hour call again. But what are your thoughts on removing these altogether from crontab:

• Full peer list
• Currently connected peers
• Full TX pool

and but those on a click button for a user to call when they want a refresh? It would get rid of quite a bit of resources for these larger calls and loads of writes to files.

[shermand100]

Fix in progress I thought I'd patched this here:

b0e1d5261b8a16152c92a2108606f8cfa6091dfb

But further testing showed that any log rotate (Monero or otherwise) broke the hard symbolic link, leaving the log file accesible by the apache server obsolete.

I'm adding this comment for context of the fix that will be committed in a few moments

[shermand100]

Create Apache alias config https://github.com/monero-ecosystem/PiNode-XMR/commit/b0e1d5261b8a16152c92a2108606f8cfa6091dfb Implementation (install and updater scripts) https://github.com/monero-ecosystem/PiNode-XMR/commit/d1ec2fbd95fbe34bcb8cc227fa129455e6188b86