Open alexandrecuer opened 1 year ago
Im not sure to be honest, this was something pb66 introduced a while back, it looks like it might be good to revisit this again and evaluate the differences.
You have a few logrotate custom files but emonhub already has its own rotation implemention, using logging.handlers.RotatingFileHandler.
You need specifically logrotate for emoncms, as it maybe does not have any rotation inside ? or am I mistaking ?
You have a few logrotate custom files
Yes because not all systems rotate their files by default!
Don't confuse logrotate
with log2ram
.
log2ram
works fine as long as logrotate
is configured correctly.
You need specifically logrotate for emoncms, as it maybe does not have any rotation inside
Correct. we rely on logrotate
to rotate the emoncms logs.
In answer to the first question, I think I asked this a while back. Key difference is that it doesn't use CRON but rather a service.timer
.
I dont confuse log2ram and logrotate but the log2ram version used by the EmonScripts does itself the confusion : the logrotate configuration of 00_olddir is done whithin log2ram install script. This is not very nice ;-(
it is better to install log2ram before apache2 and other packages.... In the scripts, the install of log2ram is done at the end of the whole process
the logrotate configuration of 00_olddir is done whithin log2ram install script.
Yes as olddir
is only relevant (for emoncms) when log2ram
is installed.
it is better to install log2ram before apache2 and other packages
Because? I cannot see it makes any difference.
[edit]
I run emoncms on an LXC. I don't use log2ram
but I do want all the logs to rotate correctly in place.
I always have problems when installing log2ram at the end of the process. Since I install it first, i face no more problem but I am now using the official log2ram apt package. For me, it makes sense that it is a better/more natural practise to install the log file system manager before the different softwares begin writing on their log files...
in the doc, it is written REBOOT before installing anything else (for example apache2)
even if it is not implicit, it implies that log2ram is installed first
I always have problems when installing log2ram at the end of the process.
This might be because the persistent journald file exists. Trystan had the same problem.
I am now using the official log2ram apt package.
Interesting, did not know that was a thing. Where from? It isn't in the RaspberryOS repo list.
For me, it makes sense that it is a better/more natural practise to install the log file system manager before the different softwares begin writing on their log files.
Fair enough - I suspect that has never been considered.
You have a few logrotate custom files but emonhub already has its own rotation implemention, using logging.handlers.RotatingFileHandler.
Yes but see https://github.com/openenergymonitor/emonhub/issues/194. Personally, I think using logrotate especially when log2ram is in use, is better than using the inbuilt log rotation as the inbuilt rotation does not respect olddir
directive.
No emoncms just sends the logs to file, no rotation.
Note that logrotate does not rotate it's own logs by default!
The original log2ram repo is there
https://github.com/azlux/log2ram
It explains how to install through apt :
https://github.com/azlux/log2ram#via-apt-recommended
It is quite instructive and there is a script to build the deb file :
https://github.com/azlux/log2ram/blob/master/build-packages.sh
I have started to use a simple script for my rotations, it is enough and i dont have the headache to maintain a custom conf for logrotate....
here is it :
#!/bin/bash
# archivage des logs inertes
# $1 dossier des log eg /var/log
# $2 software eg emoncms or emonhub
# on veut envoyer tous les fichiers .1 dans /var/log.old/software
# A automatiser avec cron :
# crontab -e
# pour exécuter cette tache tous les jours à 8h30
# 30 08 * * * /path/to/script/./archive_log.sh /var/log bios
# creating /var/log.old if it does not exist
if [ ! -d $1.old ]; then
sudo mkdir $1.old
fi
# creating /var/log.old/software if it does not exist
if [ ! -d $1.old/$2 ]; then
sudo mkdir $1.old/$2
fi
# reading the /var/log/software folder
for file in $1/$2/*
do
if [[ $file == *".1" ]]; then
fullname=$(echo $file | rev | cut -d"." -f2- | rev)
nb=$(echo $file | rev | cut -d"." -f1 | rev)
name=$(echo $fullname | rev | cut -d"/" -f1 | rev)
# relabelling .1, .2, .3... files in /var/log.old/software
# to .2, .3, .4
# limiting to 10 files max
# using tmp files so not to get stuck
for arch in $1.old/$2/$name.*
do
if [ -f $arch ]; then
fullarch=$(echo $arch | rev | cut -d"." -f2- | rev)
nbarch=$(echo $arch | rev | cut -d"." -f1 | rev)
if [[ $((nbarch)) == 10 ]]; then
sudo rm -Rf $arch
fi
if [ -f $arch ]; then
nbarch_increased=$((nbarch+1))
sudo mv $arch $fullarch.$nbarch_increased.tmp
fi
fi
done
for tmp_file in $1.old/$2/$name.*.tmp
do
if [ -f $tmp_file ]; then
real_name=$(echo $tmp_file | rev | cut -d"." -f2- | rev)
sudo mv $tmp_file $real_name
fi
done
# moving the .1 file to the /var/log.old/software folder
sudo mv $file $1.old/$2/$name.$nb
fi
done
I have started to use a simple script for my rotations
Which to me is crazy when Debian has a perfectly capable system that simply needs a config file to work. YMMV.
It explains how to install through apt :
Thanks. I was too lazy to look.
It'll be worth looking at at some point. The issue though is (as ever) migrating current users or making the update script account for the type of install in place.
I think this is an ain't broke; don't fix
Which to me is crazy when Debian has a perfectly capable system that simply needs a config file to work
What I find also crazy is to use a rotation manager (logrotate) on a sofware that already integrates its own rotation system (emonhub) :-).. What you need is not another rotation manager but just an archiving tool.
What you need is not another rotation manager but just an archiving tool.
Because it doesn't work with log2ram (creates rotated files that are then not rotated and saved by log2ram), has a limited life of rotated files and does not use olddir
. Other than that it is fine :)
@TrystanLea : the original log2ram seems to be regurlarly maintained, it is writing the old files to /var/hdd.log so I was wondering why we use a different version ?
The original log2ram now uses rsync by default
https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L47 https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L75 https://github.com/azlux/log2ram/blob/bf8f16bba6ce5a697e74af0a22a8974b08620ac7/log2ram#L10
https://github.com/azlux/log2ram/commit/1af7db2b28899a5584c01bc2d0fa17f416aac965