Random FB2.5 crashed after compiling with newer GCC/glib

[mrgleba]

As a start I am aware FB2.5 is no longer supported (we are still blocked with the migration because of the cursor stability change). I am filing this issue in case somebody knows a solution and as a reference to others with a similar problem.

We compile FB from source (with -O3) and everything went fine till Debian bullseye (gcc 10.2.1, glibc 2.31).

When repeating the same procedure on Debian bookworm (gcc 12.2, glibc 2.36) and Ubuntu jammy (gcc 11.2.0, glibc 2.35) we get random crashes /usr/local/firebird/bin/fbguard: /usr/local/firebird/bin/fbserver terminated abnormally (-1) or hangs.

We've tried -fno-tree-vectorize as suggested by a known bug in Debian bookworm but this didn't solve the issue.

We weren't able to establish if the problem lies within the compiler or the libraries as compiling with glibc 2.31 and running on glibc 2.34 and later is immpossible due to libpthread being removed/integrated directly into glibc (missing linked lib).

I'd appreciate any help looking for a cause/solution.

Have later versions of FB been successfuly tried running on those newer systems compiled with -O3?

[aafemt]

Check https://github.com/FirebirdSQL/firebird/issues/5384. Most likely it is a lifetime or destructor optimization issue.

[mrgleba]

After some trial & error I've found the gcc switch that solves the problem: -fabi-version=15

As the problems begin with gcc-11 it seems from https://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Dialect-Options.html#index-fabi-version that the problem is: Version 16, which first appeared in G++ 11, changes the mangling of alignof to be distinct from that of alignof, and dependent operator names.

[mrgleba]

Regretfuly it still crashes with the above flag 😢

[AlexPeshkoff]

Instead of trying to guess what happens I suggest you to use core dumps + gdb and send here stack traces for 2 or 3 crashes. Usually it's better to fix crash :)

[mrgleba]

That's probably where I'll start next :) Are there any tutorials how to do it on linux?

[AlexPeshkoff]

On 11/23/23 16:27, Michał Ziemski wrote:

That's probably where I'll start next :) Are there any tutorials how to do it on linux?

Try http://www.ibphoenix.com/resources/documents/search/doc_36 It's very old but if you carefully skip all about old versions it works.

[mrgleba]

Might https://ib-aid.com/articles/problem-with-libpthread-library-in-linux-installation-of-2.5 be the reason for the crashes?

[AlexPeshkoff]

On 4/12/24 11:59, Michał Ziemski wrote:

Might https://ib-aid.com/articles/problem-with-libpthread-library-in-linux-installation-of-2.5 be the reason for the crashes?

Not realted with gcc, what about glibc - definitely may be the reason. Try suggested test first of all.

[mrgleba]

This seems to be the problem. In our current environment (old debian bullseye):

Set signals
Sigenv works

On the newer system (debian bookworm):

Set signals
Segmentation fault (core dumped)

Any chance for a backport to 2.5 (even as just a commit) (a humble plea from "WELES" a years-long silver sponsor of Firebird)? We're still some work away before we can migrate to ver 5. Cursor stability introduced in 3.0 is a blocker for us as we'd need to upgrade our code to factor that in.

[AlexPeshkoff]

The main problem here is that I'm missing build/test infrastructure for FB25 (almost 15 years old compiler/etc is needed). What do you think about HQ patch? But please be aware that it's also not well tested yet.

[mrgleba]

Is the HQ patch available somwhere publicly? We could give it a try in our dev environment.

[AlexPeshkoff]

segfault.zip Please try it

[mrgleba]

Thank you. I'll give it a spin in our dev environment and report back.

[mrgleba]

After ~1 month of testing I can confirm the patch solves the issue.

In case anyone should look for the patched code, it's available at: https://github.com/mrgleba/firebird/commits/B2_5_Release