index file not being updated

[impaktor]

Since several days, elfeed doesn't update the index file. The md5sum of it is identical to an older backup I keep, and I see the same "read feeds" show up every day when I restart emacs + elfeed as "unread".

My version of elfeed is 20180121.1648, and emcas: GNU Emacs 25.3.1 (x86_64-pc-linux-gnu, GTK+ Version 3.22.26) of 2017-12-04.

I keep the backup in ~/:

[impaktor@x201][~]% ls -l index 
-rw-r--r-- 1 impaktor users 1386388 21 jan 21.15 index
[impaktor@x201][~]% ls -l .elfeed 
totalt 1360
drwxr-xr-x 2 impaktor users    4096 23 jan 21.54 data
-rw-r--r-- 1 impaktor users 1386388 23 jan 20.33 index
[impaktor@x201][~]% md5sum index 
fc2488a2558f28ed651354dea71b45f0  index
[impaktor@x201][~]% md5sum .elfeed/index 
fc2488a2558f28ed651354dea71b45f0  .elfeed/index

[skeeto]

Hmm, there are a few possibilities, though the timestamps on your files may rule out two of them:

1. Emacs isn't being shut down cleanly. Elfeed doesn't actually save after most actions. Instead it hooks Emacs' shutdown (kill-emacs-hook) to save the database just before Emacs exits. If you lose power or abruptly kill Emacs, Elfeed will lose data.

You can force a save by killing the elfeed-search buffer. You could also manually evaluate (elfeed-db-save). Then check the md5 of the index file.

2. One of your feeds is misbehaved. Sometimes feeds don't use stable entry identifiers, so Elfeed thinks they're new entries every time it fetches from the feed. This can happen when there's dynamic tracking information in the link URL (dreaded UTM parameters), or even when ads are dynamically injected into the feed. If multiple different feeds show up as "unread" all the time, then this probably isn't it.

3. The permissions on ~/.elfeed/ are screwed up and you can't write files here. The aforementioned exit hook swallows any errors, so you won't see them when it tries to save on exit. (It's reeeeally annoying to have an error prevent Emacs from exiting.) However, you will see the error if you invoke either of the manuals database saves above.

It's been long on my mental TODO list to have Elfeed save more frequently (and then fail more loudly when there's a problem), but I never got around to it yet.

Thanks for including all the version information by the way.

[impaktor]

I usually press "q" in elfeed-search buffer, and that's seemed to work the month(s) I've been using elfeed. (I might have switched from newsticker to elfeed in November, or there abuts).

1. I run Emacs in daemon mode, so it's shutdown when I power off. I think I C-x C-x the frame that's connected to it before powering off.

Evaluating (elfeed-db-save) sure had an effect on the index file, so I assume that worked (file is now bigger).

2. I notice feeds from many different sources coming back as unread. e.g. xkcd, smbc, youtube-channels, and planetemacs, and 4 other sites.

3. As you noted, permissions of folder are fine.

Also, M-x elfeed-kill-buffer also seems to update the index file, fine (I assume).

Now I'll see tomorrow if I again get a bunch of old, read, feeds as unread.

Is there anything more/else I should do?

[skeeto]

Your database may have crossed some threshold in size where Emacs couldn't write it all before the system killed it. There was no database corruption thanks to Emacs doing the standard write / fsync / rename / fsync dance (see write-region-inhibit-fsync), but the new version was completely lost.

Assuming your habits are typical, I decided to add a save upon burying the search buffer with "q" (elfeed-search-quit-window). It should now be a little more aggressive about saving the database.

[impaktor]

Assuming your habits are typical, I decided to add a save upon burying the search buffer with "q"

That was my thinking as well, or/and "g" for update key.

Your database may have crossed some threshold in size where Emacs couldn't write it all before the system killed it.

Although I don't quite see why this problem seems to affect me, as I assume my index file isn't freekishly large. I think my .elfeed folder is about 2 Mb, if I remember correctly (not on that machine now).