Having a hard time with persistent storage

[Lennart01]

Hey, I'm running into some issues with persistent storage and live monitoring. I want to constantly transfer data from my log file to the persistent storage location but also keep already saved logs. But somehow every time I launch goaccess with --real-time-html --persist --restore it doesn't constantly save to my storage location. It isn't quite clear to me whether it saves periodically or not at all. I couldn't really find anything regarding this in the doc. I'm honestly really confused rn. I managed to load all of my old log files using zcat /var/log/apache2/access.log.*.gz | goaccess -a and then goaccess --persist If you could help me out by telling me wether it will save to my data location would be great. Also does running it daemonized have an impact on it? Edit: config file is setup correctly with a persistent storage location.

[allinurl]

I'd run it once as:

# zcat access.log.*.gz | goaccess access.log.1 - -o report.html --log-format=COMBINED --persist

and then just keep adding data to the db as:

# goaccess access.log -o report.html --log-format=COMBINED --persist --restore

Every time the log rotates, access.log would be the newest and you would just keep adding the new data on top it.

Also, before you try that, I'd make sure to the delete current runs that you may have persisted before, e.g.,

# rm -rf /tmp/*.db

Give it a shot and let me know how it goes.

[Lennart01]

yeah about that. my initial question would be whether if I run it with --real-time-html and ---persist will it save only once on execution or also down the line? and thanks if this doesn't work I will definitely try that.

[Lennart01]

If --real-time-html --persist doesnt autosave I would just run it once with --realtime-html and implement a log rotate hook as described in the guide by Arnaud Rebillout. I kinda want a live monitoring with live data but also long-term data storage. Ive now managed to get my old logs pulled with the method you described. Im now running it with goaccess /var/log/apache2/access.log -o /var/www/html/downloads/report.html --persist --restore --log-format=COMBINED --real-time-html

[allinurl]

You should be fine, --real-time-html doesn't affect anything since a complete parse is done, the only difference is that when using real-time, the response to the browser is much lighter than a full blown report. The data is persisted upon exiting/closing goaccess. e.g., SIGINT - Ctrl+C

[Lennart01]

so If I run it with --real-time-html from now on it will autosave on termination. Thank you very much that saved me a lot of hastle.

[allinurl]

Happy to help :)

Closing this, feel free to reopen it if needed.