ebusd.log 7.3GB

[lukicsl]

I realised that a ran out of space on my raspbery pi. After analysis I found the reason. ebusd was generating sub-ms entries in the log file:

2019-02-10 16:30:43.962 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.963 [mqtt error] communication error: errno 22=Invalid argument
2019-02-10 16:30:43.964 [mqtt error] communication error: errno 22=Invalid argument

I have no clue why, for sure it is not a good logging strategy

[john30]

agreed, but you should look for the reason in the first place

[lukicsl]

At first place I have to make debug switchable by environment variable. I am running inside docker ebusd and mosquitto as swarm services. No clue why this situation happened. This is my entry script:

#!/bin/bash
set -e

if [ "${1#-}" != "$1" ]; then
    set -- ebusd "$@"
fi

exec "$@" |& tee -a /var/log/ebusd.log

have to look for DEBUG=true solution. For the moment I am going to switch off debugging

[john30]

btw: it has nothing to do with debug log level

[john30]

should be better with latest commit

[lukicsl]

I created a log-reader.sh:

LOGFILE="ebusd"
LOGDIR=/var/log/
LOGEXT=".log"
MaxFileSize=20000000
MaxDirSize=200000000
cnt=0

while : ; do

  read line
  cnt=$(($cnt+1))

  printf "%s\n" "$line" >> $LOGDIR$LOGFILE$LOGEXT

  if [ $cnt -gt 200 ];then   
    cnt=0
    file_size=`du -b $LOGDIR$LOGFILE$LOGEXT | tr -s '\t' ' ' | cut -d' ' -f1`
    dir_size=`du -b $LOGDIR | tr -s '\t' ' ' | cut -d' ' -f1`

    if [ $file_size -gt $MaxFileSize ];then   
        timestamp=`date +%s`
        mv $LOGDIR$LOGFILE$LOGEXT $LOGDIR$LOGFILE.$timestamp$LOGEXT
    fi
    if [ $dir_size -gt $MaxDirSize ];then   
        rm "$(ls -t $LOGDIR$LOGFILE* | tail -1)"
    fi
  fi

done

in Dockerfile:

......
RUN mkfifo /log-pipe

COPY docker-entrypoint.sh /
COPY log-reader.sh /

ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["-f", "--scanconfig"]

docker-entrypoint.sh:

#!/bin/bash
set -e

if [ "${1#-}" != "$1" ]; then
    set -- ebusd "$@"
fi

exec "$@" &> /log-pipe &
exec log-reader.sh < /log-pipe 

Now having log log rotation and watching for max size of log directory.