[Enhancement] Migration from official securecompliance/gvm image

[landonstewart]

Describe the solution you'd like Many folks, including myself, are using the official but outdated image securecompliance/gvm. It would be nice to be able to switch from that image to this more actively developed one.

When using deineagenturug/gvm:latest-data-full as a drop in replacement for securecompliance/gvm:debian-master-data there are problems with PostgreSQL. Likely because of the version differences.

A check on first start and migration would be really excellent. Failing that - a method to migrate existing data would be great.

ALSO - I'm capable and willing to clone and send pull requests if you can give me a basic overview of how the check/migration should work. At this point its not clear and I'd need to get up to speed before attempting to contribute changes.

[Dexus]

I started on this see the mention in #14

I would check the version in the database storage path, and them it need to update major version to the next major version, excepted of this behavior is all pre 10 major version.

I‘m not sure currently how much versions need to migrated so my initial Dockerfiles includes more then I think that are needed.

11to12 and 12 to 13 should work easy only from 13 to 14 needs additional steps.

If you like to contribute, you are welcome. But I need to know this, else I will do it myself hopefully by end of the next week.

regards Josef

[Dexus]

Btw. It will not work within the default container and should be done in an extra image that can reused with other installations (#14)

[landonstewart]

If you like to contribute, you are welcome. But I need to know this, else I will do it myself hopefully by end of the next week.

Hi Josef,

I tried to get going on this but won't have time before next week with the holidays and all.

[Dexus]

I started with a initial setup here: https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker/tree/db_upgrade

It will need for sure more work, maybe a other way. I try how I can fix the issues that I see.

[Dexus]

Here a working development build.

⚠️ Please make sure you have shutdown/stopped the old GVM container from securecompliance or whatever your old image might be!!!!

Once you have done, please make sure you have a backup created before you running this following steps. We will not touch the old data, but you'll know it is better to have a backup, then sorry. We create a temp copy of the Database before we starting the upgrade process to postgresql version 13.

Also this step create the right permissions for the new database. So you don't need to have a eye on this. Right after all steps are done, you can use our image deineagenturug/gvm:latest with the new database of postgresql 13.

docker pull deineagenturug/pgdb-upgrade-develop:latest

docker run -ti --rm -v "/path/to/old/securecompliance/database:/mnt/pg_old/data" -v "/path/to/new/deineagenturug/database:/mnt/pg_new/data" deineagenturug/pgdb-upgrade-develop:latest bash

Then you have to run in the container shell: /opt/setup/scripts/db_upgrade.sh

ℹ️ I hope you can enjoy the "DB Upgrade" tool and let me know how it works for you, any feedback is welcome.

/cc @markdesilva @LandonStewart

[markdesilva]

Hi @Dexus,

Thank you! Will try it out as soon as I'm done coaching my son for his mid year exams.

Best Regards, Mark

Here a working development build.

⚠️ Please make sure you have shutdown/stopped the old GVM container from securecompliance or whatever your old image might be!!!!

Once you have done, please make sure you have a backup created before you running this following steps. We will not touch the old data, but you'll know it is better to have a backup, then sorry. We create a temp copy of the Database before we starting the upgrade process to postgresql version 13.

Also this step create the right permissions for the new database. So you don't need to have a eye on this. Right after all steps are done, you can use our image deineagenturug/gvm:latest with the new database of postgresql 13.

docker pull deineagenturug/pgdb-upgrade-develop:latest

docker run -ti --rm -v "/path/to/old/securecompliance/database:/mnt/pg_old/data" -v "/path/to/new/deineagenturug/database:/mnt/pg_new/data" deineagenturug/pgdb-upgrade-develop:latest bash

Then you have to run in the container shell: /opt/setup/scripts/db_upgrade.sh

ℹ️ I hope you can enjoy the "DB Upgrade" tool and let me know how it works for you, any feedback is welcome.

/cc @markdesilva @LandonStewart

[Dexus]

Once it's tested a bit more and I get some feedback on it, it will be added to the gvm image as well. So there is no need for an extra image.

[landonstewart]

So here's what I did to test it with your instructions in mind:

1. docker-compose down
2. mv /data/gvm/database /data/gvm/database.OLD
3. $ docker run -ti --rm -v "/data/gvm/database.OLD:/mnt/pg_old/data" -v "/data/gvm/database:/mnt/pg_new/data" deineagenturug/pgdb-upgrade-develop:latest bash
4. Within the container shell executed: # /opt/setup/scripts/db_upgrade.sh (answered 'y' when asked) # exit
5. Modified my docker-compose.yml to use image: deineagenturug/gvm:latest-data-full
6. Executed: docker-compose up -d

It worked without a hitch. Thank you!

I've attached the output of the docker container during the upgrade: database_upgrade_output.txt

[Dexus]

@LandonStewart Thank you for sharing the path in more detailed steps.

Your log shows an identical output to mine. Perfect!

[netbix]

The conversion is successful but when the docker starts this error is thrown:

gvmdebian | redis: started gvmdebian | Wait for redis socket to be created... gvmdebian | Testing redis status... gvmdebian | Redis ready. gvmdebian | Creating Database folder and rights... gvmdebian | Starting PostgreSQL... gvmdebian | 2022-04-23 12:47:48,993 INFO spawned: 'postgresql' with pid 590 gvmdebian | 2022-04-23 12:47:59,063 INFO success: postgresql entered RUNNING state, process has stayed up for > than 10 seconds (startsecs) gvmdebian | postgresql: started gvmdebian | Generate SSH-HOST Keys gvmdebian | Creating Greenbone Vulnerability Manager database gvmdebian | createuser: error: creation of new role failed: ERROR: role "gvm" already exists gvmdebian | 2022-04-23 12:47:59,277 INFO exited: init (exit status 1; not expected)

[netbix]

I think i found the problem. in the new database folder these two files are missing, present in the old database folder:

-rw-r--r-- 1 syslog input 0 Apr 23 14:13 .firstrun

and

-rw-r--r-- 1 syslog input 0 Apr 23 14:13 .upgrade_to_21.4.0

[markdesilva]

Hi @LandonStewart, would it be alright to send me your docker-compose.yml please so I can take a look? I think I'm doing something wrong with the volumes.

Thank you.

So here's what I did to test it with your instructions in mind:

1. `docker-compose down`

2. `mv /data/gvm/database /data/gvm/database.OLD`

3. `$ docker run -ti --rm -v "/data/gvm/database.OLD:/mnt/pg_old/data" -v "/data/gvm/database:/mnt/pg_new/data" deineagenturug/pgdb-upgrade-develop:latest bash`

4. Within the container shell executed:
   `# /opt/setup/scripts/db_upgrade.sh` (answered 'y' when asked)
   `# exit`

5. Modified my docker-compose.yml to use `image: deineagenturug/gvm:latest-data-full`

6. Executed: `docker-compose up -d`

It worked without a hitch. Thank you!

I've attached the output of the docker container during the upgrade: database_upgrade_output.txt

[Dexus]

@netbix you previous image/tag was which one? Will look into it and fix it when I get the same result.

Okay don't need to have this info, it's right that the file is missing because it will not copied. Will fix it in the evening.

[netbix]

i migrated from: securecompliance/gvm:debian-master-data TO deineagenturug/gvm:latest-data-full

I have change my config with:

USERNAME=admin PASSWORD=admin DB_PASSWORD=xxxxxxxxxx HTTPS=true SSHD=true AUTO_SYNC=YES AUTO_SYNC_ON_START=YES TZ=Europe/Rome RELAYHOST=x.x.x.x SMTPPORT=25 OPT_PDF=1

and compose:

version: '3.4'

services:

gvmdebian: image: deineagenturug/gvm:latest-data-full hostname: gvmdebian container_name: gvmdebian restart: always ports:

• "8082:9392"
• "5432:5432"
• "2222:22" env_file:
• ./config/local.env volumes:
• ./storage/postgres-db:/opt/database
• ./storage/openvas-plugins:/var/lib/openvas/plugins
• ./storage/gvm:/var/lib/gvm
• ./storage/ssh:/etc/ssh

with those two files everything works

[Dexus]

Yes I had not thought about the lock files. Will fix it in the evening, and release a new upgrade image.

[netbix]

what is this variable for?

export NO_AUTO_UPDATE=${NO_AUTO_UPDATE:-NO}

[netbix]

thank you !

[Dexus]

what is this variable for?

export NO_AUTO_UPDATE=${NO_AUTO_UPDATE:-NO}

I hope I did not use it, had used it in a local test. It's the opposite of the AUTO_SYNC_ON_START=YES when I'm get it right from remembers.

have to say you all thank you for the feedback.

[netbix]

further problem:

scanning does not work. OpenVAS_scanner is not found

[netbix]

root@gvmdebian:/# gvmd --get-scanners root@gvmdebian:/# gvmd --get-users root@gvmdebian:/# gvmd --version Greenbone Vulnerability Manager 21.4.5 Manager DB revision 242 Copyright (C) 2009-2021 Greenbone Networks GmbH License: AGPL-3.0-or-later This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law.

[netbix]

the process seems to work:

[netbix]

I was wrong:

but why is it not detected and the scans no longer work?

[netbix]

[Dexus]

@netbix good catch, will fix it with the release 21.4.4 oder 21.4.5 the socket has changed with the merge of ospd and ospd-openvas.

[Dexus]

@netbix run sudo -u postgres psql -d gvmd -c "UPDATE public.scanners SET host='/run/ospd/ospd-openvas.sock' WHERE name='OpenVAS Default' and (host='/var/run/ospd/ospd.sock' or host='/run/ospd/ospd.sock');"

within the gvm container should fix it.

Will be done later automatically in the db upgrade script and also on the start.sh with the next release.

[markdesilva]

When upgrading the db I get this error:

lc_collate values for database "postgres" do not match: old "C.UTF-8", new "en_US.UTF-8"

If I use this db to startup the image, there are no errors, but then none of my reports, tasks, overrides, etc are there.

Another weird issue is that the CERTIFICATE and CERTIFICATE_KEY environment variables don't get recognized for some reason. The start script always executes gsad-https instead of gsad-https-owncert. I tested this by editing the /opt/setup/config/supevisord.conf for gsad-https and adding the certs into the command directly and when I restart the container then the certs loaded correctly.

Oh and on the wiki it says "Deprecated values: true,TRUE,false,FALSE" for SSHD and HTTPS. But if I use "YES", its not accepted.

Thank you.

[Dexus]

When upgrading the db I get this error:

lc_collate values for database "postgres" do not match: old "C.UTF-8", new "en_US.UTF-8"

If I use this db to startup the image, there are no errors, but then none of my reports, tasks, overrides, etc are there.

This can't work, because the DB is not get upgraded with this error. Need to check how I get the correct lc_collate and reuse them. Thank you for the report, will check and create an update for the db_upgrade.sh

Another weird issue is that the CERTIFICATE and CERTIFICATE_KEY environment variables don't get recognized for some reason. The start script always executes gsad-https instead of gsad-https-owncert. I tested this by editing the /opt/setup/config/supevisord.conf for gsad-https and adding the certs into the command directly and when I restart the container then the certs loaded correctly.

Thanks, will look into it also.

Oh and on the wiki it says "Deprecated values: true,TRUE,false,FALSE" for SSHD and HTTPS. But if I use "YES", its not accepted.

Ups was to early with my changed to the wiki. release is not yet there where this will point to.

Thank you.

Thank you <3

[markdesilva]

Thank you @Dexus!

This can't work, because the DB is not get upgraded with this error. Need to check how I get the correct lc_collate and reuse them. Thank you for the report, will check and create an update for the db_upgrade.sh

For the gsad-https-owncert issue, I was looking at the start.sh and I think its running gsad-https because its not recognizing the "YES" in the env. It is also not recognizing "TRUE" (all caps) but "true" (small letters) works.

Thank you!

[markdesilva]

@Dexus, also is it ok to add "--http-sts" to the supervisord,conf for the gsad-https and gsad-https-owncert commands to add HTTP Strict Transport Security? If not I can add it on my own, but it would be good to have.

Thank you!

[Dexus]

A new version of the upgrade image is online: docker pull deineagenturug/pgdb-upgrade-develop:latest

• fix the lock files, and reset them on the new DB version
• stop if the old and the new DB have the same version
• It will handle the ospd/openvas socket from older versions also the secure compliance solutions version. and fix it to the current used one.
• Also it handles now the old LC_COLLATE and LC_CTYPE from the old DB and use it with the initdb command for the new version.

@netbix @markdesilva would nice if you can retry the upgrade - I have currently only mobile internet access and not able to check with different images and setups. Thank you.

[netbix]

sudo -u postgres psql -d gvmd -c "UPDATE public.scanners SET host='/run/ospd/ospd-openvas.sock' WHERE name='OpenVAS Default' and (host='/var/run/ospd/ospd.sock' or host='/run/ospd/ospd.sock');"

Thank you !

Perfect !

[Dexus]

@Dexus, also is it ok to add "--http-sts" to the supervisord,conf for the gsad-https and gsad-https-owncert commands to add HTTP Strict Transport Security? If not I can add it on my own, but it would be good to have.

Thank you!

@markdesilva

To keep the thread here clean on the migration, can you open a issue for it? Thank you.

[netbix]

A new version of the upgrade image is online: docker pull deineagenturug/pgdb-upgrade-develop:latest

* fix the lock files, and reset them on the new DB version

* stop if the old and the new DB have the same version

* It will handle the ospd/openvas socket from older versions also the secure compliance solutions version. and fix it to the current used one.

* Also it handles now the old `LC_COLLATE` and `LC_CTYPE` from the old DB and use it with the `initdb` command for the new version.

@netbix @markdesilva would nice if you can retry the upgrade - I have currently only mobile internet access and not able to check with different images and setups. Thank you.

I have restore my backup DB, but ....

[netbix]

maybe this is the problem?

[markdesilva]

@Dexus thank you for the fast fix!

It’s 1.30am here now, will try when I wake up later and let you know if it works!

@netbix @markdesilva would nice if you can retry the upgrade - I have currently only mobile internet access and not able to check with different images and setups. Thank you.

I have restore my backup DB, but ....

[markdesilva]

Will do, thank you.

To keep the thread here clean on the migration, can you open a issue for it? Thank you.

[netbix]

I have correct IF but ....

calmly it will be revised a bit

[netbix]

[Dexus]

@netbix thank you and sorry that I was not double check it, a new image is uploaded with the fixes.

docker pull deineagenturug/pgdb-upgrade-develop:latest

Fixed:

• if with old_version == new_version insted of old == old
• sed with / separator and content with / don't like to work together use as separator now the # of the 2 lines where it need.

[netbix]

many many thanks !!!

[netbix]

I'm sorry but it doesn't work! calmly you will see tomorrow

[netbix]

[netbix]

[Dexus]

Thank you! Damn... 😮‍💨 today I will not able to change this, but think tomorrow I can do it.

[Dexus]

@netbix thank you and now it should work, used the now the correct config files and not the already for the temp db modified files

docker pull deineagenturug/pgdb-upgrade-develop:latest

Fixed:

• copy of already edited configs to new DB -> should not copy the temp DB config -> copies now the old config to new DB

[netbix]

Perfect !!! Now everything works! After update feed I'm running the first task with new engine. Thank you so much thank you

[netbix]

Does the file for the first download exist in your work?

[Dexus]

Does the file for the first download exist in your work?

local_multistep_build_v2.sh on devel branch line 227 is download it. So yes it's on my system and work :) or do you mean the lock file?

EDIT: It is only used on build via local_multistep_build_v2.sh and if you set DL_DATA ENV its to make the build and of the -data variants a bit faster - because if I work via mobile cell network the traffic will kill me.

[markdesilva]

Getting this error in with the new db upgrade:

Checking for presence of required libraries fatal

Full error in "loadable_libraries.txt" is :

could not load library "/usr/local/lib/libgvm-pg-server": ERROR:  could not access file "/usr/local/lib/libgvm-pg-server": No such file or directory
In database: gvmd

Whole upgrade log:

Note, selecting 'postgresql-12' instead of 'postgresql-contrib-12'
dpkg is already the newest version (1.20.9).
libglib2.0-0 is already the newest version (2.66.8-1).
locales is already the newest version (2.31-13+deb11u3).
libgnutls30 is already the newest version (3.7.1-5).
libgpgme11 is already the newest version (1.14.0-1+b2).
libhiredis0.14 is already the newest version (0.14.1-1).
libical3 is already the newest version (3.0.9-2).
libnet1 is already the newest version (1.1.6+dfsg-3.1).
libpcap0.8 is already the newest version (1.10.0-2).
libssh-gcrypt-4 is already the newest version (0.9.5-1+deb11u1).
libxml2 is already the newest version (2.9.10+dfsg-6.7+deb11u1).
nano is already the newest version (5.4-2).
libldap-2.4-2 is already the newest version (2.4.57+dfsg-3).
libradcli4 is already the newest version (1.2.11-1+b2).
libuuid1 is already the newest version (2.36.1-8+deb11u1).
zip is already the newest version (3.0-12).
postgresql-common is already the newest version (238.pgdg110+1).
The following additional packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu clang-11 lib32gcc-s1 lib32stdc++6 libasan6 libbinutils libc-dev-bin libc6-dev libc6-i386 libclang-common-11-dev libclang-cpp11 libclang1-11 libcrypt-dev libctf-nobfd0 libctf0
  libffi-dev libgc1 libgcc-10-dev libitm1 liblsan0 libncurses-dev libnsl-dev libobjc-10-dev libobjc4 libpfm4 libquadmath0 libssl-dev libstdc++-10-dev libtinfo-dev libtirpc-dev libtsan0 libubsan1 libyaml-0-2 libz3-dev linux-libc-dev
  llvm-11 llvm-11-dev llvm-11-runtime llvm-11-tools python3-pygments python3-yaml
Suggested packages:
  binutils-doc clang-11-doc glibc-doc manpages-dev ncurses-doc postgresql-doc-14 libssl-doc libstdc++-10-doc llvm-11-doc postgresql-doc-12 python-pygments-doc ttf-bitstream-vera
Recommended packages:
  libomp-11-dev manpages manpages-dev libc-devtools binfmt-support sysstat
The following NEW packages will be installed:
  binutils binutils-common binutils-x86-64-linux-gnu clang-11 lib32gcc-s1 lib32stdc++6 libasan6 libbinutils libc-dev-bin libc6-dev libc6-i386 libclang-common-11-dev libclang-cpp11 libclang1-11 libcrypt-dev libctf-nobfd0 libctf0
  libffi-dev libgc1 libgcc-10-dev libitm1 liblsan0 libncurses-dev libnsl-dev libobjc-10-dev libobjc4 libpfm4 libpq-dev libquadmath0 libssl-dev libstdc++-10-dev libtinfo-dev libtirpc-dev libtsan0 libubsan1 libyaml-0-2 libz3-dev
  linux-libc-dev llvm-11 llvm-11-dev llvm-11-runtime llvm-11-tools postgresql-12 postgresql-client-12 postgresql-server-dev-12 python3-pygments python3-yaml
The following packages will be upgraded:
  xz-utils
1 upgraded, 47 newly installed, 0 to remove and 3 not upgraded.
Need to get 101 MB of archives.
After this operation, 593 MB of additional disk space will be used.

Preconfiguring packages ...
(Reading database ...
Preparing to unpack .../00-xz-utils_5.2.5-2.1~deb11u1_amd64.deb ...
Unpacking xz-utils (5.2.5-2.1~deb11u1) over (5.2.5-2) ...
Selecting previously unselected package binutils-common:amd64.
Preparing to unpack .../01-binutils-common_2.35.2-2_amd64.deb ...
Unpacking binutils-common:amd64 (2.35.2-2) ...
Selecting previously unselected package libbinutils:amd64.
Preparing to unpack .../02-libbinutils_2.35.2-2_amd64.deb ...
Unpacking libbinutils:amd64 (2.35.2-2) ...
Selecting previously unselected package libctf-nobfd0:amd64.
Preparing to unpack .../03-libctf-nobfd0_2.35.2-2_amd64.deb ...
Unpacking libctf-nobfd0:amd64 (2.35.2-2) ...
Selecting previously unselected package libctf0:amd64.
Preparing to unpack .../04-libctf0_2.35.2-2_amd64.deb ...
Unpacking libctf0:amd64 (2.35.2-2) ...
Selecting previously unselected package binutils-x86-64-linux-gnu.
Preparing to unpack .../05-binutils-x86-64-linux-gnu_2.35.2-2_amd64.deb ...
Unpacking binutils-x86-64-linux-gnu (2.35.2-2) ...
Selecting previously unselected package binutils.
Preparing to unpack .../06-binutils_2.35.2-2_amd64.deb ...
Unpacking binutils (2.35.2-2) ...
Selecting previously unselected package libclang-cpp11.
Preparing to unpack .../07-libclang-cpp11_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang-cpp11 (1:11.0.1-2) ...
Selecting previously unselected package libitm1:amd64.
Preparing to unpack .../08-libitm1_10.2.1-6_amd64.deb ...
Unpacking libitm1:amd64 (10.2.1-6) ...
Selecting previously unselected package libasan6:amd64.
Preparing to unpack .../09-libasan6_10.2.1-6_amd64.deb ...
Unpacking libasan6:amd64 (10.2.1-6) ...
Selecting previously unselected package liblsan0:amd64.
Preparing to unpack .../10-liblsan0_10.2.1-6_amd64.deb ...
Unpacking liblsan0:amd64 (10.2.1-6) ...
Selecting previously unselected package libtsan0:amd64.
Preparing to unpack .../11-libtsan0_10.2.1-6_amd64.deb ...
Unpacking libtsan0:amd64 (10.2.1-6) ...
Selecting previously unselected package libubsan1:amd64.
Preparing to unpack .../12-libubsan1_10.2.1-6_amd64.deb ...
Unpacking libubsan1:amd64 (10.2.1-6) ...
Selecting previously unselected package libquadmath0:amd64.
Preparing to unpack .../13-libquadmath0_10.2.1-6_amd64.deb ...
Unpacking libquadmath0:amd64 (10.2.1-6) ...
Selecting previously unselected package libgcc-10-dev:amd64.
Preparing to unpack .../14-libgcc-10-dev_10.2.1-6_amd64.deb ...
Unpacking libgcc-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package libc-dev-bin.
Preparing to unpack .../15-libc-dev-bin_2.31-13+deb11u3_amd64.deb ...
Unpacking libc-dev-bin (2.31-13+deb11u3) ...
Selecting previously unselected package linux-libc-dev:amd64.
Preparing to unpack .../16-linux-libc-dev_5.10.106-1_amd64.deb ...
Unpacking linux-libc-dev:amd64 (5.10.106-1) ...
Selecting previously unselected package libcrypt-dev:amd64.
Preparing to unpack .../17-libcrypt-dev_1%3a4.4.18-4_amd64.deb ...
Unpacking libcrypt-dev:amd64 (1:4.4.18-4) ...
Selecting previously unselected package libtirpc-dev:amd64.
Preparing to unpack .../18-libtirpc-dev_1.3.1-1_amd64.deb ...
Unpacking libtirpc-dev:amd64 (1.3.1-1) ...
Selecting previously unselected package libnsl-dev:amd64.
Preparing to unpack .../19-libnsl-dev_1.3.0-2_amd64.deb ...
Unpacking libnsl-dev:amd64 (1.3.0-2) ...
Selecting previously unselected package libc6-dev:amd64.
Preparing to unpack .../20-libc6-dev_2.31-13+deb11u3_amd64.deb ...
Unpacking libc6-dev:amd64 (2.31-13+deb11u3) ...
Selecting previously unselected package libstdc++-10-dev:amd64.
Preparing to unpack .../21-libstdc++-10-dev_10.2.1-6_amd64.deb ...
Unpacking libstdc++-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package libgc1:amd64.
Preparing to unpack .../22-libgc1_1%3a8.0.4-3_amd64.deb ...
Unpacking libgc1:amd64 (1:8.0.4-3) ...
Selecting previously unselected package libobjc4:amd64.
Preparing to unpack .../23-libobjc4_10.2.1-6_amd64.deb ...
Unpacking libobjc4:amd64 (10.2.1-6) ...
Selecting previously unselected package libobjc-10-dev:amd64.
Preparing to unpack .../24-libobjc-10-dev_10.2.1-6_amd64.deb ...
Unpacking libobjc-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package libc6-i386.
Preparing to unpack .../25-libc6-i386_2.31-13+deb11u3_amd64.deb ...
Unpacking libc6-i386 (2.31-13+deb11u3) ...
Selecting previously unselected package lib32gcc-s1.
Preparing to unpack .../26-lib32gcc-s1_10.2.1-6_amd64.deb ...
Unpacking lib32gcc-s1 (10.2.1-6) ...
Selecting previously unselected package lib32stdc++6.
Preparing to unpack .../27-lib32stdc++6_10.2.1-6_amd64.deb ...
Unpacking lib32stdc++6 (10.2.1-6) ...
Selecting previously unselected package libclang-common-11-dev.
Preparing to unpack .../28-libclang-common-11-dev_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang-common-11-dev (1:11.0.1-2) ...
Selecting previously unselected package libclang1-11.
Preparing to unpack .../29-libclang1-11_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang1-11 (1:11.0.1-2) ...
Selecting previously unselected package clang-11.
Preparing to unpack .../30-clang-11_1%3a11.0.1-2_amd64.deb ...
Unpacking clang-11 (1:11.0.1-2) ...
Selecting previously unselected package libffi-dev:amd64.
Preparing to unpack .../31-libffi-dev_3.3-6_amd64.deb ...
Unpacking libffi-dev:amd64 (3.3-6) ...
Selecting previously unselected package libncurses-dev:amd64.
Preparing to unpack .../32-libncurses-dev_6.2+20201114-2_amd64.deb ...
Unpacking libncurses-dev:amd64 (6.2+20201114-2) ...
Selecting previously unselected package libpfm4:amd64.
Preparing to unpack .../33-libpfm4_4.11.1+git32-gd0b85fb-1_amd64.deb ...
Unpacking libpfm4:amd64 (4.11.1+git32-gd0b85fb-1) ...
Selecting previously unselected package libssl-dev:amd64.
Preparing to unpack .../34-libssl-dev_1.1.1n-0+deb11u1_amd64.deb ...
Unpacking libssl-dev:amd64 (1.1.1n-0+deb11u1) ...
Selecting previously unselected package libpq-dev.
Preparing to unpack .../35-libpq-dev_14.2-1.pgdg110+1_amd64.deb ...
Unpacking libpq-dev (14.2-1.pgdg110+1) ...
Selecting previously unselected package libtinfo-dev:amd64.
Preparing to unpack .../36-libtinfo-dev_6.2+20201114-2_amd64.deb ...
Unpacking libtinfo-dev:amd64 (6.2+20201114-2) ...
Selecting previously unselected package libyaml-0-2:amd64.
Preparing to unpack .../37-libyaml-0-2_0.2.2-1_amd64.deb ...
Unpacking libyaml-0-2:amd64 (0.2.2-1) ...
Selecting previously unselected package libz3-dev:amd64.
Preparing to unpack .../38-libz3-dev_4.8.10-1_amd64.deb ...
Unpacking libz3-dev:amd64 (4.8.10-1) ...
Selecting previously unselected package llvm-11-runtime.
Preparing to unpack .../39-llvm-11-runtime_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-runtime (1:11.0.1-2) ...
Selecting previously unselected package llvm-11.
Preparing to unpack .../40-llvm-11_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11 (1:11.0.1-2) ...
Selecting previously unselected package python3-pygments.
Preparing to unpack .../41-python3-pygments_2.7.1+dfsg-2.1_all.deb ...
Unpacking python3-pygments (2.7.1+dfsg-2.1) ...
Selecting previously unselected package python3-yaml.
Preparing to unpack .../42-python3-yaml_5.3.1-5_amd64.deb ...
Unpacking python3-yaml (5.3.1-5) ...
Selecting previously unselected package llvm-11-tools.
Preparing to unpack .../43-llvm-11-tools_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-tools (1:11.0.1-2) ...
Selecting previously unselected package llvm-11-dev.
Preparing to unpack .../44-llvm-11-dev_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-dev (1:11.0.1-2) ...
Selecting previously unselected package postgresql-client-12.
Preparing to unpack .../45-postgresql-client-12_12.10-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-client-12 (12.10-1.pgdg110+1) ...
Selecting previously unselected package postgresql-12.
Preparing to unpack .../46-postgresql-12_12.10-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-12 (12.10-1.pgdg110+1) ...
Selecting previously unselected package postgresql-server-dev-12.
Preparing to unpack .../47-postgresql-server-dev-12_12.10-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-server-dev-12 (12.10-1.pgdg110+1) ...
Setting up libz3-dev:amd64 (4.8.10-1) ...
Setting up llvm-11-runtime (1:11.0.1-2) ...
Setting up libyaml-0-2:amd64 (0.2.2-1) ...
Setting up binutils-common:amd64 (2.35.2-2) ...
Setting up linux-libc-dev:amd64 (5.10.106-1) ...
Setting up libctf-nobfd0:amd64 (2.35.2-2) ...
Setting up python3-yaml (5.3.1-5) ...
Setting up libffi-dev:amd64 (3.3-6) ...
Setting up libasan6:amd64 (10.2.1-6) ...
Setting up libclang-cpp11 (1:11.0.1-2) ...
Setting up python3-pygments (2.7.1+dfsg-2.1) ...
Setting up postgresql-client-12 (12.10-1.pgdg110+1) ...
Setting up libtirpc-dev:amd64 (1.3.1-1) ...
Setting up libpfm4:amd64 (4.11.1+git32-gd0b85fb-1) ...
Setting up xz-utils (5.2.5-2.1~deb11u1) ...
Setting up libquadmath0:amd64 (10.2.1-6) ...
Setting up libssl-dev:amd64 (1.1.1n-0+deb11u1) ...
Setting up libclang1-11 (1:11.0.1-2) ...
Setting up libgc1:amd64 (1:8.0.4-3) ...
Setting up libubsan1:amd64 (10.2.1-6) ...
Setting up libnsl-dev:amd64 (1.3.0-2) ...
Setting up libcrypt-dev:amd64 (1:4.4.18-4) ...
Setting up libc6-i386 (2.31-13+deb11u3) ...
Setting up libbinutils:amd64 (2.35.2-2) ...
Setting up postgresql-12 (12.10-1.pgdg110+1) ...
Creating new PostgreSQL cluster 12/main ...
/usr/lib/postgresql/12/bin/initdb -D /var/lib/postgresql/12/main --auth-local peer --auth-host md5
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/12/main ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    pg_ctlcluster 12 main start

invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libc-dev-bin (2.31-13+deb11u3) ...
Setting up liblsan0:amd64 (10.2.1-6) ...
Setting up libitm1:amd64 (10.2.1-6) ...
Setting up libtsan0:amd64 (10.2.1-6) ...
Setting up libctf0:amd64 (2.35.2-2) ...
Setting up llvm-11-tools (1:11.0.1-2) ...
Setting up libgcc-10-dev:amd64 (10.2.1-6) ...
Setting up llvm-11 (1:11.0.1-2) ...
Setting up libpq-dev (14.2-1.pgdg110+1) ...
Setting up libobjc4:amd64 (10.2.1-6) ...
Setting up lib32gcc-s1 (10.2.1-6) ...
Setting up lib32stdc++6 (10.2.1-6) ...
Setting up libobjc-10-dev:amd64 (10.2.1-6) ...
Setting up libclang-common-11-dev (1:11.0.1-2) ...
Setting up libc6-dev:amd64 (2.31-13+deb11u3) ...
Setting up binutils-x86-64-linux-gnu (2.35.2-2) ...
Setting up libncurses-dev:amd64 (6.2+20201114-2) ...
Setting up libstdc++-10-dev:amd64 (10.2.1-6) ...
Setting up llvm-11-dev (1:11.0.1-2) ...
Setting up binutils (2.35.2-2) ...
Setting up clang-11 (1:11.0.1-2) ...
Setting up postgresql-server-dev-12 (12.10-1.pgdg110+1) ...
Setting up libtinfo-dev:amd64 (6.2+20201114-2) ...
Processing triggers for postgresql-common (238.pgdg110+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Processing triggers for libc-bin (2.31-13+deb11u3) ...
Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.
removed '/var/lib/apt/lists/apt.postgresql.org_pub_repos_apt_dists_bullseye-pgdg_InRelease'
removed '/var/lib/apt/lists/apt.postgresql.org_pub_repos_apt_dists_bullseye-pgdg_main_binary-amd64_Packages.lz4'
removed directory '/var/lib/apt/lists/auxfiles'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-backports_InRelease'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-backports_main_binary-amd64_Packages.lz4'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-updates_InRelease'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye-updates_main_binary-amd64_Packages.lz4'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_InRelease'
removed '/var/lib/apt/lists/deb.debian.org_debian_dists_bullseye_main_binary-amd64_Packages.lz4'
removed '/var/lib/apt/lists/lock'
removed directory '/var/lib/apt/lists/partial'
removed '/var/lib/apt/lists/security.debian.org_debian-security_dists_bullseye-security_InRelease'
removed '/var/lib/apt/lists/security.debian.org_debian-security_dists_bullseye-security_main_binary-amd64_Packages.lz4'
Copy old PG_DATA and chown to process user
waiting for server to start...... done
server started
init new db version 13
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /mnt/pg_new/data ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    /usr/lib/postgresql/13/bin/pg_ctl -D /mnt/pg_new/data/ -l logfile start

waiting for server to start.... done
server started
waiting for server to shut down.... done
server stopped
waiting for server to shut down.... done
server stopped
Start checking of Postgresql pg_upgrade from 12 to 13
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for system-defined composite types in user tables  ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for presence of required libraries                 fatal

Your installation references loadable libraries that are missing from the
new installation.  You can add these libraries to the new installation,
or remove the functions using them from the old installation.  A list of
problem libraries is in the file:
    loadable_libraries.txt

Failure, exiting

Thank you.

[Dexus]

@markdesilva did you pull the latest image version? It sounds like you did not. I had this error only in old versions and it should already fixed.

EDIT: Also the path of the lib is different... Which image you used before from SCS?