22.4 Update and Upgrade issues

[immauss]

If you have issues with the new 22.4, please add here. As of today, this is VERY beta. Upgrades of existin databases may not work. You have been warned. I'll add here as it cleans up so follow to stay up to date.

-Scott

[thoefkens-fielmann]

Hello!

When trying to build with base-rebuild, I am getting these issues:

#0 787.8 Setting up mono-gac (6.8.0.105+dfsg-3.2) ...
#0 787.9 update-alternatives: using /usr/bin/gacutil to provide /usr/bin/cli-gacutil (global-assembly-cache-tool) in auto mode
#0 787.9 Setting up mono-runtime-sgen (6.8.0.105+dfsg-3.2) ...
#0 787.9 Setting up mono-runtime (6.8.0.105+dfsg-3.2) ...
#0 788.0 update-alternatives: using /usr/bin/mono to provide /usr/bin/cli (cli) in auto mode
#0 788.0 Setting up libmono-corlib4.5-cil (6.8.0.105+dfsg-3.2) ...
#0 788.7 Setting up libglib2.0-cil (2.12.40-3) ...
#0 790.9 
#0 790.9 =================================================================
#0 790.9        External Debugger Dump:
#0 790.9 =================================================================
#0 790.9 mono_gdb_render_native_backtraces not supported on this platform, unable to find gdb or lldb
#0 790.9 qemu: uncaught target signal 6 (Aborted) - core dumped
#0 790.9 Aborted
#0 791.4 
#0 791.4 =================================================================
#0 791.4        External Debugger Dump:
#0 791.4 =================================================================
#0 791.4 mono_gdb_render_native_backtraces not supported on this platform, unable to find gdb or lldb
#0 791.4 qemu: uncaught target signal 6 (Aborted) - core dumped
#0 791.4 Aborted
#0 791.4 E: installing Assembly /usr/share/cli-common/policies.d/libglib2.0-cil/policy.2.10.glib-sharp.dll failed
#0 791.4 E: Installation of policy.2.10.glib-sharp with /usr/share/cli-common/runtimes.d/mono failed
#0 791.4 dpkg: error processing package libglib2.0-cil (--configure):
#0 791.4  installed libglib2.0-cil package post-installation script subprocess returned error exit status 1
#0 791.4 dpkg: dependency problems prevent configuration of libglib2.0-cil-dev:
#0 791.4  libglib2.0-cil-dev depends on libglib2.0-cil (= 2.12.40-3); however:
#0 791.4   Package libglib2.0-cil is not configured yet.
#0 791.4 
#0 791.4 dpkg: error processing package libglib2.0-cil-dev (--configure):
#0 791.4  dependency problems - leaving unconfigured
#0 791.4 Processing triggers for libc-bin (2.31-13+deb11u3) ...
#0 791.5 Processing triggers for tex-common (6.16) ...
#0 792.9 Running updmap-sys. This may take some time... done.
#0 795.9 Running mktexlsr /var/lib/texmf ... done.
#0 797.6 Building format(s) --all.
#0 797.6        This may take some time... done.
#0 857.6 Errors were encountered while processing:
#0 857.6  libglib2.0-cil
#0 857.6  libglib2.0-cil-dev
#0 857.8 E: Sub-process /usr/bin/dpkg returned an error code (1)
------
Dockerfile:6
--------------------
   5 |     #COPY sources.list /etc/apt/
   6 | >>> RUN bash /scripts/install-deps.sh && \
   7 | >>>  date > /ovasbase-build-date
   8 |     ENTRYPOINT ["/bin/bash"] 
--------------------
error: failed to solve: process "/bin/sh -c bash /scripts/install-deps.sh &&  date > /ovasbase-build-date" did not complete successfully: exit code: 100

Any ideas what might cause this?

Thanks a lot in advance!

Thomas

[thoefkens-fielmann]

When testing immauss/openvas:22.4-beta, I get the following issue when running the container the first time:

server started
Checking for existing DB
Running first start configuration...
NOTICE:  relation "vt_severities" already exists, skipping
NOTICE:  relation "vt_severities" already exists, skipping
NOTICE:  relation "vt_severities" already exists, skipping
Migrating the database to the latest version if needed.
2022-08-25 07:24:56.003 UTC [5461] ERROR:  function hosts_contains(text,text) is not a member of extension "pg-gvm"
2022-08-25 07:24:56.003 UTC [5461] DETAIL:  An extension is not allowed to replace an object that it does not own.
2022-08-25 07:24:56.003 UTC [5461] STATEMENT:  CREATE EXTENSION IF NOT EXISTS "pg-gvm"

(gvmd:5458): md manage-WARNING **: 07:24:56.007: sql_exec_internal: PQexec failed: ERROR:  function hosts_contains(text,text) is not a member of extension "pg-gvm"
DETAIL:  An extension is not allowed to replace an object that it does not own.
 (7)

(gvmd:5458): md manage-WARNING **: 07:24:56.010: sql_exec_internal: SQL: CREATE EXTENSION IF NOT EXISTS "pg-gvm"

(gvmd:5458): md manage-WARNING **: 07:24:56.010: sqlv: sql_exec_internal failed

[immauss]

@thoefkens-fielmann That script is really written more for my environment. I've never really tried to make it portable. So YMMV.

And the other errors for the container, you can figure that out, you'll be my hero. I've had a thread over at the greenbone community forum for several days now with no response. Know anyone with some serious postgresql XP?

https://community.greenbone.net/t/database-migration-from-21-04-22-04/

[immauss]

As a short status update:

The build works great with a fresh new database. But upgrades from old databases from 21.4 fail horribly. I'm going to push a working image shortly with a fresh 22.4 DB as the current still has the db from 21.4.

[kjake]

oof, yeah, just found my installation broken :(

NOTICE:  relation "vt_severities" already exists, skipping
Migrating the database to the latest version if needed.
(gvmd:47): md manage-WARNING **: 17:33:55.996: sql_exec_internal: PQexec failed: ERROR:  column "date" does not exist
LINE 1: ...WHERE task = $1 AND scan_run_status = 1  ORDER BY date DESC ...
                                                             ^
 (7)
(gvmd:47): md manage-WARNING **: 17:33:55.996: sql_exec_internal: SQL: CREATE OR REPLACE FUNCTION task_last_report (integer) RETURNS integer AS $$  SELECT id FROM reports WHERE task = $1 AND scan_run_status = 1  ORDER BY date DESC LIMIT 1;$$ LANGUAGE SQL;

[immauss]

@kjake
Oof. Which tag is that and when did you pull it? -Scott

[kjake]

Just removed and pulled latest and it is largely the same:

NOTICE:  relation "vt_severities" already exists, skipping
NOTICE:  relation "vt_severities" already exists, skipping
Migrating the database to the latest version if needed.
(gvmd:47): md manage-WARNING **: 20:22:11.305: sql_exec_internal: PQexec failed: ERROR:  column "date" does not exist
LINE 1: ...WHERE task = $1 AND scan_run_status = 1  ORDER BY date DESC ...
                                                             ^
 (7)
(gvmd:47): md manage-WARNING **: 20:22:11.305: sql_exec_internal: SQL: CREATE OR REPLACE FUNCTION task_last_report (integer) RETURNS integer AS $$  SELECT id FROM reports WHERE task = $1 AND scan_run_status = 1  ORDER BY date DESC LIMIT 1;$$ LANGUAGE SQL;
(gvmd:47): md manage-WARNING **: 20:22:11.305: sqlv: sql_exec_internal failed

[immauss]

Can you pull the gvmd.log from /var/log/gvm/ and send that to me? Or at least the last 200 lines of it if it's really large.

I'm hoping ..... you didn't get caught in the mess from earlier today, but it sounds like maybe you did. There was a point where the 22.4-beta was accidentally pushed to 'latest' It was there for less than an hour . . . . . but it was in the middle of the day. In order to upgrade the DB from 21.4 to 22.4, I had to write an sql script that removes ALL of the functions, extensions, views and operators specific to gvmd, and then start a gvmd --migrate.

See the drama unfold here: https://forum.greenbone.net/t/database-migration-from-21-04-22-04/12834/11

If you did pull that image, then it wiped the functions from you DB, and your DB has probably been upgraded to 22.4. I appologize. Totally my fault, so I promise I'll do whatever I can to help you work this out.

The best course will likely be to upgrade to 22.4, but .... I only just got the scripts working today for the upgrade and I've only run it against a fresh clean DB.

For now. MAKE A BACKUP. Before you do anything else.

I'm going to do some accelerated testing with 22.4 and I'll let you know how it goes. But please send the gvmd.log. I should be able to confirm with that.

-Scott

I've learned a great deal over the last few days about the postgres DB schema from gvmd and I'm sure if we have to, I can gronk you DB back into line if we need to.

[immauss]

OK ... I've done some limited testing with 22.4.1-beta. It looks good. Scanning works Downloaded a PDF of scan results. upgraded my production DB with no issues.

So you "should" be able to use it with you DB from earlier. BUT !!!!!!! MAKE A BACKUP BEFORE YOU TRY IT !!!

Just copy the entire data volume to be sure.

and make sure you leave the SKIPSYNC false the first time. It will take a while, but if you don't you won't have the feeds for the notuscanner. (new bit from greenbone.)

Again ... really sorry about the hiccup today.

-Scott

[kjake]

Hey, no problem, things happen, and that's what backups are for :)

But, you did give me a hint on how to get up-and-running without restoring from backup: I used the 22.4.1-beta tag, and it started right up!

I do have logs for you though: https://send.kjake.net/download/e33cdb15e7ff0798/#AEzugZjxv_JjERCeowsdgA

I'll DM you the password. edit: oh, your Twitter DMs aren't open. I'm @kjake over there too. msg me for the password, or your preferred OOB way to get the logs. edit2: Successfully ran a scan task with this beta tag

[immauss]

22.4 seems to working well at this point, so I'm closing this thread.