search

fluent / fluent-package-builder

td-agent (Fluentd) Building and Packaging System
Apache License 2.0
22 stars 23 forks source link

deb: fix failure on purge (td-agent v4) #564

Open kenhys opened 11 months ago

kenhys commented 11 months ago

According to https://github.com/fluent/fluentd-website/pull/266, purge failure is reported.

vagrant@vagrant:~$ apt list --installed | grep fluen

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

fluent-package/now 5.0.0-1 amd64 [installed,local]
fluentd-apt-source/focal,now 2020.8.25-1 all [installed]
vagrant@vagrant:~$ apt list --installed | grep td-agent

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

vagrant@vagrant:~$ sudo apt purge td-agent
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  td-agent*
0 upgraded, 0 newly installed, 1 to remove and 194 not upgraded.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] 
(Reading database ... 50480 files and directories currently installed.)
Purging configuration files for td-agent (4.5.0-1) ...
td-agent:x:113:117::/var/lib/td-agent:/usr/sbin/nologin
userdel: user td-agent is currently used by process 21510
dpkg: error processing package td-agent (--purge):
 installed td-agent package post-removal script subprocess returned error exit status 8
Errors were encountered while processing:
 td-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)
kenhys commented 11 months ago

Verified with the following situation:

 dpkg -l|grep -E "td-agent|fluent"
ii  fluent-package                        5.0.0-1                                 amd64        Treasure Agent: A data collector for Treasure
 Data
rc  td-agent                              4.5.0-1                                 amd64        Treasure Agent: A data collector for Treasure
 Data

Then execute sudo apt purge td-agent.

td-agent v4.5.0 is assumed that td-agent service must be stopped before purging td-agent package.

. /usr/share/debconf/confmodule

if [ "$1" = "purge" ]; then
        rm -f /etc/default/td-agent
        dpkg-statoverride --list /etc/td-agent > /dev/null && \
                dpkg-statoverride --remove /etc/td-agent
        rm -f /etc/td-agent/td-agent.conf
        rm -rf /etc/td-agent
        dpkg-statoverride --list /var/run/td-agent > /dev/null && \
                dpkg-statoverride --remove /var/run/td-agent
        rm -f /var/run/td-agent/*
        rm -rf /var/run/td-agent
        dpkg-statoverride --list /var/log/td-agent > /dev/null && \
                dpkg-statoverride --remove /var/log/td-agent
        rm -rf /var/log/td-agent/buffer
        rm -rf /var/log/td-agent/*
        rm -rf /var/log/td-agent

        getent passwd td-agent && userdel -r td-agent
fi
kenhys commented 11 months ago

On jammy, td-agent is marked as rc. on the contract to it, td-agent is marked as ii on focal.

jammy:

$ dpkg -l |grep td-agent
rc  td-agent                              4.5.0-1                                 amd64        Treasure Agent: A data collector for Treasure
 Data
vagrant@ubuntu2204:~$ sudo apt purge td-agent
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  td-agent*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] y
(Reading database ... 120911 files and directories currently installed.)
Purging configuration files for td-agent (4.5.0-1) ...
td-agent:x:114:121::/var/lib/td-agent:/usr/sbin/nologin
userdel: user td-agent is currently used by process 39797
dpkg: error processing package td-agent (--purge):
 installed td-agent package post-removal script subprocess returned error exit status 8
Errors were encountered while processing:
 td-agent
E: Sub-process /usr/bin/dpkg returned an error code (1)

focal:

$ sudo apt purge td-agent
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  td-agent*
0 upgraded, 0 newly installed, 1 to remove and 1 not upgraded.
After this operation, 15.4 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 121430 files and directories currently installed.)
Removing td-agent (5.0.0-1) ...
(Reading database ... 121427 files and directories currently installed.)
Purging configuration files for td-agent (5.0.0-1) ...
kenhys commented 11 months ago

If you purge td-agent 5.0.0, no problem.

$ sudo apt purge td-agent
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following packages will be REMOVED:
  td-agent*
0 upgraded, 0 newly installed, 1 to remove and 0 not upgraded.
After this operation, 15.4 kB disk space will be freed.
Do you want to continue? [Y/n] y
(Reading database ... 120972 files and directories currently installed.)
Removing td-agent (5.0.0-1) ...
(Reading database ... 120969 files and directories currently installed.)
Purging configuration files for td-agent (5.0.0-1) ...
daipom commented 11 months ago

Hmm, can we really purge v4 safely?

If a user upgrades to v5 from v4, I suppose that the v4 package should not be purged.

The focal environment where the package is upgraded to v5 from v4:

$ dpkg -l td-agent
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-==================================================
rc  td-agent       4.5.0-1      amd64        Treasure Agent: A data collector for Treasure Data

$ dpkg -L td-agent
/etc
/etc/logrotate.d
/etc/logrotate.d/td-agent
/etc/td-agent
/etc/td-agent/td-agent.conf
/opt

$ ls -la /etc/td-agent
lrwxrwxrwx 1 root root 11 Aug  8 08:24 /etc/td-agent -> /etc/fluent

$ ls -la /etc/fluent/
drwxr-xr-x  3 td-agent td-agent 4096 Aug  8 08:24 .
drwxr-xr-x 99 root     root     4096 Aug  8 08:24 ..
lrwxrwxrwx  1 root     root       25 Aug  8 08:24 fluentd.conf -> /etc/fluent/td-agent.conf
drwxr-xr-x  2 root     root     4096 Aug  8 08:24 plugin
-rw-r--r--  1 root     root     2382 Aug  8 08:24 td-agent.conf

For example, I suppose that we should not purge the config file: /etc/td-agent/td-agent.conf because it is still used by v5.

In addition, I wonder if this v4 postrm script would break the migrated environment...

https://github.com/fluent/fluent-package-builder/blob/ac9ca53c81a2ff2a72c67cb18d17aa6decb48d8f/td-agent/templates/package-scripts/td-agent/deb/postrm#L7-L24

daipom commented 11 months ago

I confirmed we can purge td-agent v4 without the error if we stop the service in advance. However, as I am concerned, files are deleted unexpectedly, corrupting the environment.

The focal environment where the package is upgraded to v5 from v4:

$ sudo systemctl stop fluentd
$ sudo apt purge td-agent
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages will be REMOVED:
  td-agent*
0 upgraded, 0 newly installed, 1 to remove and 194 not upgraded.
After this operation, 0 B of additional disk space will be used.
Do you want to continue? [Y/n] 
(Reading database ... 50480 files and directories currently installed.)
Purging configuration files for td-agent (4.5.0-1) ...
td-agent:x:113:117::/var/lib/td-agent:/usr/sbin/nologin
userdel: group td-agent is the primary group of another user and is not removed.
userdel: td-agent mail spool (/var/mail/td-agent) not found
# It seems to be no problem that the user `td-agent` is removed.
vagrant@vagrant:~$ getent passwd td-agent
(empty)

vagrant@vagrant:~$ getent passwd _fluentd
_fluentd:x:113:117::/var/lib/fluent:/usr/sbin/nologin

# The config file and the symbolic link are removed.
vagrant@vagrant:~$ la /etc/td-agent
ls: cannot access '/etc/td-agent': No such file or directory

vagrant@vagrant:~$ ls -la /etc/fluent
drwxr-xr-x  3 _fluentd td-agent 4096 Aug  8 09:09 .
drwxr-xr-x 99 root     root     4096 Aug  8 09:09 ..
lrwxrwxrwx  1 root     root       25 Aug  8 09:07 fluentd.conf -> /etc/fluent/td-agent.conf
drwxr-xr-x  2 root     root     4096 Aug  8 09:07 plugin

# The log/buffer files and the symbolic link are cleared.
vagrant@vagrant:~$ ls -la /var/log/td-agent
ls: cannot access '/var/log/td-agent': No such file or directory

vagrant@vagrant:~$ ls -la /var/log/fluent/
drwxr-xr-x 2 _fluentd td-agent 4096 Aug  8 09:09 .
drwxrwxr-x 8 root     syslog   4096 Aug  8 09:09 ..