bbb-install script needs to handle configuration file updates during upgrades

PhMemmel commented 3 years ago

Running bbb-install.sh on an existent 2.3.0 installation fails the first time because kurento config files have been modified and dpkg apparently expects user input to decide what it should do.

Running bbb-install.sh a second time makes it run through. However, even after the second run, the new config file has not been installed. As mentioned in the release notes there was a renaming of an option in WebRtcEndpoint.conf.ini for example. The admin has to manually merge WebRtcEndpoint.conf.ini.dpkg-dist into WebRtcEndpoint.conf.ini.

As far as I can see kurento still accepts niceAgentIceTcp config key or at least this wrong option in WebRtcEndpoint.conf.ini doesn't prevent kurento from running and apparently working, but this probably should be fixed.

EDIT: BTW: The existing of WebRtcEndpoint.conf.ini.dpkg-dist makes kurento log errors because of the unknown suffix as well. But this is a minor issue.

Output of first run of bbb-install.sh, I left the non BBB related lines which appeared in between the BBB related ones so one can see the issue of dpkg probably waiting for user input on STDIN:

Configuration file '/etc/kurento/modules/kurento/WebRtcEndpoint.conf.ini'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** WebRtcEndpoint.conf.ini (Y/I/N/O/D/Z) [default=N] ? Setting up kms-filters (                                                                                                                                                             6.16.0.bionic.20210503130759.4.2ae49cb) ...
Setting up grub-efi-amd64 (2.04-1ubuntu44) ...
Installing for x86_64-efi platform.
Installation finished. No error reported.
Sourcing file `/etc/default/grub'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.0-73-generic
Found initrd image: /boot/initrd.img-5.4.0-73-generic
Found linux image: /boot/vmlinuz-5.4.0-72-generic
Found initrd image: /boot/initrd.img-5.4.0-72-generic
Adding boot menu entry for EFI firmware configuration
done
Setting up kurento-media-server (6.16.0-0kurento1.18.04) ...

Configuration file '/etc/default/kurento-media-server'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** kurento-media-server (Y/I/N/O/D/Z) [default=N] ? dpkg: error processing pack                                                                                                                                                             age kurento-media-server (--configure):
 end of file on stdin at conffile prompt
Setting up grub-efi-amd64-signed (1.167~18.04.1+2.04-1ubuntu44) ...
Installing for x86_64-efi platform.
Installation finished. No error reported.
dpkg: dependency problems prevent configuration of bbb-webrtc-sfu:
 bbb-webrtc-sfu depends on kurento-media-server; however:
  Package kurento-media-server is not configured yet.

dpkg: error processing package bbb-webrtc-sfu (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup erro                                                                                                                                                             r from a previous failure.
dpkg: dependency problems prevent configuration of bbb-html5:
 bbb-html5 depends on bbb-webrtc-sfu; however:
  Package bbb-webrtc-sfu is not configured yet.

dpkg: error processing package bbb-html5 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of bbb-config:
 bbb-config depends on bbb-html5; however:
  Package bbb-html5 is not configured yet.

dpkg: error processing package bbb-config (--configure):
 dependency problems - leaving unconfigured
Processing triggers for systemd (237-3ubuntu10.47) ...
No apport report written because the error message indicates its a followup erro                                                                                                                                                             r from a previous failure.
No apport report written because MaxReports is reached already
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for dbus (1.12.2-1ubuntu1.2) ...
Processing triggers for ureadahead (0.100.0-21) ...
Processing triggers for install-info (6.5.0.dfsg.1-2) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
Processing triggers for initramfs-tools (0.130ubuntu3.11) ...
update-initramfs: Generating /boot/initrd.img-5.4.0-73-generic
I: The initramfs will attempt to resume from /dev/sdb3
I: (UUID=361f394d-af5d-4a85-84a3-8dc85527bcf1)
I: Set the RESUME variable to override this.
Processing triggers for shim-signed (1.37~18.04.8+15+1552672080.a4a1fbe-0ubuntu2) ...
Errors were encountered while processing:
 kurento-media-server
 bbb-webrtc-sfu
 bbb-html5
 bbb-config
E: Sub-process /usr/bin/dpkg returned an error code (1)
RTNETLINK answers: File exists
'universe' distribution component is already enabled for all sources.
Hit:1 https://deb.nodesource.com/node_12.x bionic InRelease
Hit:2 https://download.docker.com/linux/ubuntu bionic InRelease
Hit:3 http://ppa.launchpad.net/bigbluebutton/support/ubuntu bionic InRelease
Ign:4 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 InRelease
Hit:5 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 Release
Hit:6 http://de.archive.ubuntu.com/ubuntu bionic InRelease
Hit:7 http://de.archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:8 http://ppa.launchpad.net/certbot/certbot/ubuntu bionic InRelease
Hit:9 http://de.archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:10 http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic InRelease
Hit:11 http://de.archive.ubuntu.com/ubuntu bionic-security InRelease
Hit:12 http://ppa.launchpad.net/rmescandon/yq/ubuntu bionic InRelease
Hit:13 https://ubuntu.bigbluebutton.org/bionic-23 bigbluebutton-bionic InRelease
Reading package lists... Done
Warning: apt-key output should not be parsed (stdout is not a terminal)
Hit:1 https://deb.nodesource.com/node_12.x bionic InRelease
Hit:2 http://ppa.launchpad.net/bigbluebutton/support/ubuntu bionic InRelease
Ign:3 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 InRelease
Hit:4 https://repo.mongodb.org/apt/ubuntu bionic/mongodb-org/4.2 Release
Hit:5 https://download.docker.com/linux/ubuntu bionic InRelease
Hit:6 http://de.archive.ubuntu.com/ubuntu bionic InRelease
Hit:7 http://de.archive.ubuntu.com/ubuntu bionic-updates InRelease
Hit:8 http://ppa.launchpad.net/certbot/certbot/ubuntu bionic InRelease
Hit:9 http://de.archive.ubuntu.com/ubuntu bionic-backports InRelease
Hit:10 http://ppa.launchpad.net/libreoffice/ppa/ubuntu bionic InRelease
Hit:11 http://de.archive.ubuntu.com/ubuntu bionic-security InRelease
Hit:12 http://ppa.launchpad.net/rmescandon/yq/ubuntu bionic InRelease
Hit:14 https://ubuntu.bigbluebutton.org/bionic-23 bigbluebutton-bionic InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following packages will be REMOVED:
  golang-docker-credential-helpers libsecret-1-0 libsecret-common python-asn1crypto python-backports.ssl-match-hostname python-cached-property python-certifi python-cffi-backend python-chardet python-cryptography python-docker
  python-dockerpty python-dockerpycreds python-docopt python-enum34 python-funcsigs python-functools32 python-idna python-ipaddress python-jsonschema python-mock python-openssl python-pbr python-pkg-resources python-requests python-six
  python-texttable python-urllib3 python-websocket python-yaml
0 upgraded, 0 newly installed, 30 to remove and 0 not upgraded.
4 not fully installed or removed.
After this operation, 9468 kB disk space will be freed.
(Reading database ... 228102 files and directories currently installed.)
Removing python-docker (2.5.1-1) ...
Removing python-dockerpycreds (0.2.1-1) ...
Removing golang-docker-credential-helpers (0.5.0-2) ...
Removing libsecret-1-0:amd64 (0.18.6-1) ...
Removing libsecret-common (0.18.6-1) ...
Removing python-openssl (17.5.0-1ubuntu1) ...
Removing python-cryptography (2.1.4-1ubuntu1.4) ...
Removing python-asn1crypto (0.24.0-1) ...
Removing python-backports.ssl-match-hostname (3.5.0.1-1) ...
Removing python-cached-property (1.3.1-1) ...
Removing python-requests (2.18.4-2ubuntu0.1) ...
Removing python-certifi (2018.1.18-2) ...
Removing python-cffi-backend (1.11.5-1) ...
Removing python-chardet (3.0.4-1) ...
Removing python-dockerpty (0.4.1-1) ...
Removing python-docopt (0.6.2-1build1) ...
Removing python-enum34 (1.1.6-2) ...
Removing python-jsonschema (2.6.0-2) ...
update-alternatives: using /usr/bin/python3-jsonschema to provide /usr/bin/jsonschema (jsonschema) in auto mode
Removing python-mock (2.0.0-3) ...
Removing python-funcsigs (1.0.2-4) ...
Removing python-functools32 (3.2.3.2-3) ...
Removing python-idna (2.6-1) ...
Removing python-ipaddress (1.0.17-1) ...
Removing python-pbr (3.1.1-3ubuntu3) ...
update-alternatives: using /usr/bin/python3-pbr to provide /usr/bin/pbr (pbr) in auto mode
Removing python-pkg-resources (39.0.1-2) ...
Removing python-urllib3 (1.22-1ubuntu0.18.04.2) ...
Removing python-websocket (0.44.0-0ubuntu2) ...
Removing python-six (1.11.0-2) ...
Removing python-texttable (0.9.1-1) ...
Removing python-yaml (3.12-1build2) ...
Setting up kurento-media-server (6.16.0-0kurento1.18.04) ...

Configuration file '/etc/default/kurento-media-server'
 ==> Modified (by you or by a script) since installation.
 ==> Package distributor has shipped an updated version.
   What would you like to do about it ?  Your options are:
    Y or I  : install the package maintainer's version
    N or O  : keep your currently-installed version
      D     : show the differences between the versions
      Z     : start a shell to examine the situation
 The default action is to keep your current version.
*** kurento-media-server (Y/I/N/O/D/Z) [default=N] ? dpkg: error processing package kurento-media-server (--configure):
 end of file on stdin at conffile prompt
dpkg: dependency problems prevent configuration of bbb-webrtc-sfu:
 bbb-webrtc-sfu depends on kurento-media-server; however:
  Package kurento-media-server is not configured yet.

dpkg: error processing package bbb-webrtc-sfu (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of bbb-html5:
 bbb-html5 depends on bbb-webrtc-sfu; however:
  Package bbb-webrtc-sfu is not configured yet.

dpkg: error processing package bbb-html5 (--configure):
 dependency problems - leaving unconfigured
dpkg: dependency problems prevent configuration of bbb-config:
 bbb-config depends on bbb-html5; however:
  Package bbb-html5 is not configured yet.

dpkg: error processing package bbb-config (--configure):
 dependency problems - leaving unconfigured
No apport report written because the error message indicates its a followup error from a previous failure.
No apport report written because the error message indicates its a followup error from a previous failure.
No apport report written because MaxReports is reached already
Processing triggers for man-db (2.8.3-2ubuntu0.1) ...
Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
Errors were encountered while processing:
 kurento-media-server
 bbb-webrtc-sfu
 bbb-html5
 bbb-config
E: Sub-process /usr/bin/dpkg returned an error code (1)

EmmyGraugans commented 3 years ago

The apt-get dist-upgrade is missing something similar to this:

apt-get -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confnew" dist-upgrade

(Alternatively --force-confold, and grab the output in order to warn at the end of the run that something needs to be looked at, because default configuration-files were changed)

Assuming that something like apply-conf.sh and/or ansible/shef/puppet will add local configuration in the end again, --force-confnew should be ok however...

EmmyGraugans commented 3 years ago

It was just pointed out to me, this is not the first time this happened, see https://github.com/bigbluebutton/bbb-install/issues/330 – there should be something added to bbb-install.sh to deal with configuration changes, independently if they occurred because of packaging-changes as in #330, or if they occured because someone manually edited the configuration files.

wolbernd commented 3 years ago

A possible workaround might be to set the environment variable DEBIAN_FRONTENDto noninteractive.

According to the manpage (http://manpages.ubuntu.com/manpages/bionic/man7/debconf.7.html) this should answer all interactive dialogues with the default answer. However I was not able to test this as I already upgraded all my BBB Servers to 2.3.1 and thus can't test an upgrade any more.

I will check it out once 2.3.2 is released.

hex-m commented 2 years ago

Upgrading from 2.3.17 to 2.4.2 gave me two conflicts. My understanding is, that the *.dpkg-dist are the versions that should actually be used.

# diff /etc/bigbluebutton/nginx/web /etc/bigbluebutton/nginx/web.dpkg-dist 
10,11c10,15
<                       # Workaround IE refusal to set cookies in iframe
<                       add_header P3P 'CP="No P3P policy available"';
---
>                     # Workaround IE refusal to set cookies in iframe
>                     add_header P3P 'CP="No P3P policy available"';
>                     if ($bbb_loadbalancer_node) {
>                         add_header 'Access-Control-Allow-Origin' $bbb_loadbalancer_node always;
>                         add_header 'Access-Control-Allow-Credentials' 'true' always;
>                     }
16c20,28
<                       proxy_pass         http://127.0.0.1:8090;
---
>                       # Grails can't handle CORS OPTION preflight requests correctly -> lets do this in nginx
>                       if ($request_method = 'OPTIONS') {
>                               add_header 'Access-Control-Allow-Origin' $bbb_loadbalancer_node always;
>                               add_header 'Access-Control-Allow-Credentials' 'true' always;
>                               add_header 'Content-Type' 'text/plain; charset=utf-8';
>                               add_header 'Content-Length' 0;
>                               return 204;
>                       }
>                       proxy_pass         http://127.0.0.1:8090;
21c33,37
<                       add_header P3P 'CP="No P3P policy available"';
---
>                       add_header P3P 'CP="No P3P policy available"';
>                       if ($bbb_loadbalancer_node) {
>                               add_header 'Access-Control-Allow-Origin' $bbb_loadbalancer_node always;
>                               add_header 'Access-Control-Allow-Credentials' 'true' always;
>                       }
59a76,78
>                       if ($bbb_loadbalancer_node) {
>                               add_header 'Access-Control-Allow-Origin' $bbb_loadbalancer_node always;
>                       }
93a113,120
>               location = /bigbluebutton/connection/legacyCheckAuthorization {
>                       internal;
>                       proxy_pass               http://127.0.0.1:8090;
>                       proxy_pass_request_body  off;
>                       proxy_set_header         Content-Length "";
>                       proxy_set_header         X-Original-URI $request_uri;
>               }
> 
104a132,134
>             if ($bbb_loadbalancer_node) {
>                 add_header 'Access-Control-Allow-Origin' $bbb_loadbalancer_node always;
>             }

# diff /etc/cron.daily/bigbluebutton.dpkg-dist /etc/cron.daily/bigbluebutton
34c34
< find /var/bigbluebutton/ -maxdepth 1 -type d -name "*-[0-9]*" -mtime +$history -exec rm -rf '{}' +
---
> find /var/bigbluebutton/ -maxdepth 1 -type d -name "*-*" -mtime +$history -exec rm -rf '{}' +
37c37
< # Delete streams from Kurento and mediasoup older than N days
---
> # Delete streams in kurento older than N days
39,53c39,45
< kurento_dir=/var/kurento/
< mediasoup_dir=/var/mediasoup/
< 
< remove_stale_sfu_raw_files() {
<   for app in recordings screenshare; do
<       app_dir="${1}${app}"
<       if [[ -d $app_dir ]]; then
<           find "$app_dir" -name "*.mkv" -o -name "*.webm" -mtime +"$history" -delete
<           find "$app_dir" -type d -empty -mtime +"$history" -exec rmdir '{}' +
<       fi
<   done
< }
< 
< remove_stale_sfu_raw_files "$kurento_dir"
< remove_stale_sfu_raw_files "$mediasoup_dir"
---
> for app in recordings screenshare; do
>       app_dir=/var/kurento/$app
>       if [[ -d $app_dir ]]; then
>               find $app_dir -name "*.mkv" -o -name "*.webm" -mtime +$history -delete
>               find $app_dir -type d -empty -mtime +$history -exec rmdir '{}' +
>       fi
> done

bigbluebutton / bbb-install

bbb-install script needs to handle configuration file updates during upgrades #376