search

miniflux / v2

Minimalist and opinionated feed reader
https://miniflux.app
Apache License 2.0
6.94k stars 726 forks source link

apt upgrade: when both postgresql and miniflux are upgraded, miniflux fails to start #1154

Open elboulangero opened 3 years ago

elboulangero commented 3 years ago

Here are the logs of a fresh apt upgrade on my machine (Debian buster):

# apt full-upgrade 
[...]
The following packages will be upgraded:
  apt apt-transport-https apt-utils bind9-host exim4 exim4-base exim4-config exim4-daemon-light goaccess
  libapt-inst2.0 libapt-pkg5.0 libbind9-161 libdns-export1104 libdns1104 libisc-export1100 libisc1100
  libisccc161 libisccfg163 liblwres161 liblz4-1 libpq5 miniflux postgresql-11 postgresql-client-11
24 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
[...]
Get:4 https://apt.miniflux.app  miniflux 2.0.30 [4,184 kB]
[...]
Get:23 http://deb.debian.org/debian-security buster/updates/main amd64 postgresql-client-11 amd64 11.12-0+deb10u1 [1,409 kB]
Get:24 http://deb.debian.org/debian-security buster/updates/main amd64 postgresql-11 amd64 11.12-0+deb10u1 [14.1 MB]
[...]
Preparing to unpack .../16-miniflux_2.0.30_amd64.deb ...
Unpacking miniflux (2.0.30) over (2.0.29) ...
Preparing to unpack .../17-postgresql-client-11_11.12-0+deb10u1_amd64.deb ...
Unpacking postgresql-client-11 (11.12-0+deb10u1) over (11.11-0+deb10u1) ...
Preparing to unpack .../18-postgresql-11_11.12-0+deb10u1_amd64.deb ...
Unpacking postgresql-11 (11.12-0+deb10u1) over (11.11-0+deb10u1) ...
[...]

Setting up miniflux (2.0.30) ...
Job for miniflux.service failed because the control process exited with error code.
See "systemctl status miniflux.service" and "journalctl -xe" for details.

Setting up postgresql-client-11 (11.12-0+deb10u1) ...
Setting up liblwres161:amd64 (1:9.11.5.P4+dfsg-5.1+deb10u5) ...
Setting up postgresql-11 (11.12-0+deb10u1) ...
[...]
Building PostgreSQL dictionaries from installed myspell/hunspell packages...

And here are the corresponding systemd logs:

May 28 03:56:15 my-server miniflux[1016]: [INFO] Shutting down the process...
May 28 03:56:15 my-server systemd[1]: Stopping Miniflux Feed Reader...
May 28 03:56:15 my-server miniflux[1016]: [INFO] Process gracefully stopped 
May 28 03:56:15 my-server systemd[1]: miniflux.service: Succeeded.
May 28 03:56:15 my-server systemd[1]: Stopped Miniflux Feed Reader. 
[...]
May 28 03:56:18 my-server systemd[1]: Stopping PostgreSQL Cluster 11-main...
May 28 03:56:18 my-server systemd[1]: postgresql@11-main.service: Succeeded.
May 28 03:56:18 my-server systemd[1]: Stopped PostgreSQL Cluster 11-main.
[...]
May 28 03:56:26 my-server systemd[1]: Starting Miniflux Feed Reader...
May 28 03:56:26 my-server miniflux[5345]: [FATAL] Unable to connect to the database: dial tcp 127.0.0.1:5432: connect: connection refused 
May 28 03:56:26 my-server systemd[1]: miniflux.service: Main process exited, code=exited, status=1/FAILURE
May 28 03:56:26 my-server systemd[1]: miniflux.service: Failed with result 'exit-code'.
May 28 03:56:26 my-server systemd[1]: Failed to start Miniflux Feed Reader. 
May 28 03:56:26 my-server systemd[1]: miniflux.service: Service RestartSec=100ms expired, scheduling restart.
May 28 03:56:26 my-server systemd[1]: miniflux.service: Scheduled restart job, restart counter is at 1.
May 28 03:56:26 my-server systemd[1]: Stopped Miniflux Feed Reader. 
May 28 03:56:26 my-server systemd[1]: Starting Miniflux Feed Reader...
May 28 03:56:26 my-server miniflux[5473]: [FATAL] Unable to connect to the database: dial tcp 127.0.0.1:5432: connect: connection refused 
May 28 03:56:26 my-server systemd[1]: miniflux.service: Main process exited, code=exited, status=1/FAILURE
May 28 03:56:26 my-server systemd[1]: miniflux.service: Failed with result 'exit-code'.
May 28 03:56:26 my-server systemd[1]: Failed to start Miniflux Feed Reader. 
May 28 03:56:26 my-server systemd[1]: miniflux.service: Service RestartSec=100ms expired, scheduling restart.
May 28 03:56:26 my-server systemd[1]: miniflux.service: Scheduled restart job, restart counter is at 2.
May 28 03:56:26 my-server systemd[1]: Stopped Miniflux Feed Reader. 
[...]

## error above happens 6 times total

[...]

May 28 03:56:28 my-server systemd[1]: Starting PostgreSQL Cluster 11-main...
May 28 03:56:31 my-server systemd[1]: Started PostgreSQL Cluster 11-main.
May 28 03:56:32 my-server systemd[1]: Reloading.

So what we see clearly with those logs is that systemd tries to start miniflux BEFORE it starts postgresql. The ordering between both is not correct.

Let's look at this file: https://github.com/miniflux/v2/blob/master/packaging/systemd/miniflux.service

[Unit]
Description=Miniflux
After=network.target postgresql.service

Hmmm, so I'm not really sure that it would fix the issue, but I would rather do:

[Unit]
Description=Miniflux
After=network.target postgresql.service
Requires=postgresql.service

Requires makes it so that miniflux won't be started if postgresql is not up: it's a hard requirement. Seems to make sense here, since without postgresql, miniflux can't work and is not functional AFAIK.

But I can't be sure that such a change would fix the issue at hand here.

Any thought?

fguillot commented 3 years ago

Requires makes it so that miniflux won't be started if postgresql is not up: it's a hard requirement. Seems to make sense here, since without postgresql, miniflux can't work and is not functional AFAIK.

It's pretty common these days to run the database on a different machine. I don't know what would be Systemd behavior if Postgresql is not installed voluntary on the same host.

Perhaps Wants is a better fit: https://www.freedesktop.org/software/systemd/man/systemd.unit.html#Wants=

It's worth testing.

elboulangero commented 3 years ago

I reached out on IRC to understand better. So it's not really an issue with the systemd service, and the After= is enough for ordering from the systemd point of view. BTW, at boot, it all works, because both postgresql and miniflux are pulled into multi-user.target.wants, and then systemd can order them properly.

The issue here is that during an apt upgrade transaction, apt will remove and install packages separately, and run their various scripts (preinst, postinst, etc...). The various systemd stop and start commands are done on each package individually. It's not one big "systemd start" with all the services that were upgraded, instead it's one "systemd start" per service. So systemd can't order anything. The order for doing things is instead decided by apt, according to package relations.

So a possible solution would be for miniflux to Depend on postgresql. That will make apt do things in the right order, and it will restart the miniflux service after postgresql.

But that doesn't go well with the use-case you mentioned, ie. having the database on a different machine.

I'm not sure what's the right solution but I'll keep thinking about it. I'll keep this ticket updated.

Thx @mbiebl for the explanations on IRC