epam / Indigo

Universal cheminformatics toolkit, utilities and database search tools
http://lifescience.opensource.epam.com
Apache License 2.0
315 stars 105 forks source link

Bingo in PostgreSQL, make database terminated abnormally easily #1456

Open Pandaaaa906 opened 9 months ago

Pandaaaa906 commented 9 months ago

Steps to Reproduce

  1. PostgreSQL 12.6, compiled by Visual C++ build 1914, 64-bit
  2. Bingo 1.9.1.587-g0e3666ae7
  3. 32GB Memory just for pg

i got a table "substance"(import from pubchem), more than 40 million records. bingo will easily make pg break down, even for simple SQL like this:

SELECT * FROM "substance"
WHERE iso_smiles @ ('Cc1cc(Cl)nnc1N', 'ALL')::bingo.exact;
server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.

in the pg log:

2023-12-22 09:54:26.261 HKT [19104] LOG:  server process (PID 12960) was terminated by exception 0xC0000005
2023-12-22 09:54:26.261 HKT [19104] DETAIL:  Failed process was running: SELECT * FROM "substance"

    WHERE iso_smiles @ ('Cc1cc(Cl)nnc1N', 'ALL')::bingo.exact
2023-12-22 09:54:26.261 HKT [19104] HINT:  See C include file "ntstatus.h" for a description of the hexadecimal value.
2023-12-22 09:54:26.262 HKT [19104] LOG:  terminating any other active server processes
2023-12-22 09:54:26.266 HKT [19452] WARNING:  terminating connection because of crash of another server process
Pandaaaa906 commented 9 months ago

once the database breakdown the index usally seem corrupted, and will cause another breakdown...

error: bingo buffer: internal error: corrupted block 8148 data len is -8

i have to drop index and recreate index, when the table become massive, it's painful

Pandaaaa906 commented 9 months ago

Seem to be same and old issue: https://github.com/epam/Indigo/issues/108 https://github.com/ggasoftware/indigo/issues/15

Pandaaaa906 commented 8 months ago

@AlexanderSavelyev Any idea?

jank9201 commented 3 weeks ago

once the database breakdown the index usally seem corrupted, and will cause another breakdown...

error: bingo buffer: internal error: corrupted block 8148 data len is -8

i have to drop index and recreate index, when the table become massive, it's painful

Your Index is broken, it seems that the Index could be rewrited by the server. I wrote a patch for postgres-15.7, it seems to be ok, but it should be test more (range partition table still has problem), the attachment is the patch( master Indigo 12.5), if you would like to test it, Please make backup before testing! diff.txt