firebird-automations
commented
15 years ago Commented by: @hvlad
There is no memory leak. Each substring(blob...) creates a temporaty blob which is released only at transaction end. Each such blob occupied in memory 1 database page amount of bytes. Your code creates a lot of temporary blob's (100000). So, you need 400MB of memory just for this blobs (if your page size is 4KB).
In Firebird 2.5 temporary blobs stored on disk (in temporary files) so memory requirement are much lower.
In this case i see that engine could understand that temporary blob, created by substring, can be released immediately after implicit CAST to varchar.
As for FB 2.1 i recommend to lower amount of temporary blobs created within one transaction to reduce memory consumption.
Submitted by: Calin Pirtea (pcalin)
There seems to be a severe memmory leak in Firebird 2.1.2 and 2.1.3 RC2 related to using substring or conversion between blobs and varchar.
I'm using the latest Firebird 2.1.3.18185 and running the following script:
create or alter procedure test_crash2_FB2_1 as begin end;
create or alter procedure test_crash_FB2_1 as begin end;
recreate table tbl_test_crash_FB2_1 ( PK_Field integer not null primary key , test_memo blob (65535, 1));
create generator gen_test_crash_FB2_1;
set generator gen_test_crash_FB2_1 to 0;
commit;
set term ^;
create or alter procedure test_crash_FB2_1 as declare my_loop_count integer; begin my_loop_count = 100000; while (my_loop_count > 0) do begin insert into tbl_test_crash_FB2_1 (PK_Field, test_memo) values (gen_id(gen_test_crash_FB2_1, 1), 'asd fa dsf asdf ksdfjhgklsjhgfdklj shfdgkj hsjkgfd hklsjgfd h klsjhfdgkl sjhdfgkl jhskgfdj h');
end end^
create or alter procedure test_crash2_FB2_1 as declare my_loop_count integer; declare my_test_memo blob (65535, 1); declare my_pk integer; declare my_test_varchar varchar(4096); begin execute procedure test_crash_FB2_1; my_loop_count = 100000; while (my_loop_count > 0) do begin for select PK_Field, test_memo from tbl_test_crash_FB2_1 into :my_pk, :my_test_memo do begin my_test_varchar = substring(:my_test_memo from 1 for 4096); end my_loop_count = my_loop_count - 1; end end^
commit^
set term ;^
execute procedure test_crash2_FB2_1;
commit;
I used a virtual machine with 128 MB RAM and 128 MB VirtualMemory. After a few seconds the server crashes. The production environment where this was reported has 4096MB Ram and 4096MB Virtual Memory and it takes over 8 hours to crash the server. We are in desperate need of a fix since one of our largest customer is experiencing this DoS. At this stage we have no workarounds for the problem.
Thank you in advance,
Calin Pirtea Communicare Systems Pty Ltd.