log2ram JOURNAL_AWARE=true feature is not able to preserve any journal logs across boot

[ioogithub]

Tested with systemd 255 on ubuntu 24.04 and 22.04 version 1.7.2

Journal logs are not saved and restored after a system reboot.

To test:

1. Clear the journal with journal --rotate and journalctl --vacuum-time=1s.
2. Reboot
3. Run journalctl --list-boots and observe 1 boot entry is recorded.
4. Reboot
5. Run journalctl--list-boots and observe only the last 1 entry is recorded. All journal logs before the most recent boot are lost.
6. Double chech: Run journalctl and observe the log entries start only from the most recent boot.

Journalctl -t log2ram showes rsync copy records from after the boot including /journal/* files but nothing before the boot as those logs are lost.

Is this by design? I can't see log2ram being useful if it wipes out the logs after every reboot. The only change to log2ram was to increase size to 2560M.

It looks like this could be a bug because the log2ram.conf that was installed has double entries for the JOURNAL_AWARE=true setting:

# Should log2ram consider journald and do a log rotate before copying the log files back? Please note that for this
# rsync is mandatory. Also make sure that you have configured SystemMaxUse in journald.conf, because the size of this
# ram folder is will probably not be as large as journald will use by default
JOURNALD_AWARE=true

# Set to 'true' to enable log rotation for journald logs before syncing. 
# Note: 'rsync' must be used for this feature. 
# Ensure 'SystemMaxUse' is configured in 'journald.conf' 
#  - (To limit journald’s disk usage to be smaller than the log2ram RAM allocation).
JOURNALD_AWARE=true

Was this part of the app modified recently? If so it does not work.

[ioogithub]

Tested a few more times, removed --purged log2ram and reinstalled.

Here is what is happening. After a boot log2ram saves the journal entries from the first boot before it was installed and the last previous boot only. All other journal entries are deleted.

So if I boot 5 times 8n journalctl I would see -1 and 0:

-1 = Boot 1 0 = Boot 5 (the most recent boot).

All entries for boot 2, 3, and 4 are missing.

I can easily troubleshoot what log2ram is doing before the boot because of course all the logs are gone.

Definately not working as expected.

[ioogithub]

I am not sure but I think this could be a similar issue to: https://github.com/azlux/log2ram/issues/219

Also this issue was closed as being fixed: https://github.com/azlux/log2ram/issues/222 but it appears to also be similar.

Is log2ram confirmed as not working on ubuntu at all?

[ioogithub]

Looking deeper into this it appears that in the /var/log/journal directory the logs are still there but logs from the previous boot are corrupted with a ~ after the filename. So it looks like the problem is indeed this line:

JOURNALD_AWARE=true

log2ram is corrupting the journal when it tries to rotate.

I think it is a similar issue to this which was marked as completed but I think the issue has not been fixed: https://github.com/azlux/log2ram/issues/185.

I have tested on two systems ubuntu 22.04 and 24.04, same result. I believe the corrupted journal is why all logs are missing from before a reboot.

[ioogithub]

Discussion of the specific issue is here: https://github.com/azlux/log2ram/pull/200

[azlux]

Hi,

The double JOURNAL_AWARE don't make any issue. Since it's minor, I didn't release any new version to fix that even if it's already fix into master branch.

The option JOURNALD_AWARE allow log2ram to make a journalctl rotation before sync the /var/log folder : https://github.com/azlux/log2ram/blob/master/log2ram#L29 The code force rsync to include only journals with a @ : https://github.com/azlux/log2ram/blob/master/log2ram#L44

Let's try to investigate in this ticket instead of spreading into multiple issue.

[azlux]

I suspect systemd-journald.service starting before finishing the HDD->MEMORY sync. Can you check that ? You can get the full starting services order with the command : systemd-analyze plot > stating.svg this generate a big svg file.

[azlux]

Can you check also the last commit https://github.com/azlux/log2ram/commit/83774d470ba9ed1d2ab286a390fee65b28272d34 , it's maybe link to your issue.