FirebirdSQL / firebird

Firebird server, client and tools
https://www.firebirdsql.org/
1.19k stars 204 forks source link

Memory leak when executing a lot of different queries and StatementTimeout > 0 #8085

Closed pavel-zotov closed 1 month ago

pavel-zotov commented 1 month ago

Make firebird.conf empty and then add into it the single line:

StatementTimeout = 7200

Restart FB service. Run some tool that can show memory consumprion per process, e.g. ProcessExplorer (on Windows). Try following example (it will run infinitely so one need to cancel it via Ctrl-Break):

set list on;
set bail on;
shell del e:\temp\tmp4test.fdb 2>nul;
create database 'localhost:e:\temp\tmp4test.fdb' user sysdba password 'masterkey';

select *
from rdb$config where rdb$config_name = 'StatementTimeout';

set term ^;
execute block as
    declare i int = 0;
    declare res int;
begin
    while (1 = 1) do
    begin
        -- execute statement ( 'select rand() from rdb$database' ) -- [1] constant
        -- execute statement ( 'select ' || 1 || ' from rdb$database' ) -- [2] constant
        execute statement ( 'select ' || rand() || ' from rdb$database' ) -- [3] growth
        into res;
        i = i + 1;
    end
end
^
set term ;^

Watch into details for firebird process in ProcessExplorer (swich to 'performance graph'). You will see somth like: image

No such problem if we comment out parameter 'StatementTimeout' in the firebird.conf.

Checked on 6.0.0.315.

PS. I was unable to determine the exact value of the threshold of the number of unique expressions, after exceeding which memory growth begins. I checked MOD(i, N) function (instead of rand()) where N had different values. It seemed firstly that N is ~16, but then i get results with constant memory consumption for greater values (i.e. for N = 17, 33, 67, 129 etc).

hvlad commented 1 month ago

I was unable to determine the exact value of the threshold of the number of unique expressions, after exceeding which memory growth begins. I checked MOD(i, N) function (instead of rand()) where N had different values. It seemed firstly that N is ~16, but then i get results with constant memory consumption for greater values (i.e. for N = 17, 33, 67, 129 etc).

16 is the number of statements cached by external connection. I.e. statements was reused and no memory leak could be detected, while it actually there - near 40 bytes per statement.

hvlad commented 1 month ago

BTW, you don't have to change firebird.conf to reproduce it. It is enough to execute SET STATEMENT TIMEOUT <N> before the EXECUTE BLOCK, with N > 0.