epam / Indigo

Universal cheminformatics toolkit, utilities and database search tools
http://lifescience.opensource.epam.com
Apache License 2.0
291 stars 100 forks source link

Certain concurrent queries crash Bingo #113

Open dan2097 opened 6 years ago

dan2097 commented 6 years ago

I'm using a version built from source on the 16th November 2017 against PostgreSQL 9.6. When doing concurrent queries against a Bingo index a minority of queries cause Bingo to crash (running the same queries sequentially seems to work). As an example an exact search for 'CCCCCCCN' usually elicited the issue but not a substructure search or a similarity search for this compound. An exact search for 'CCCCCCC' also works.

Unfortunately I can't reproduce the issue on a database of a few thousand chemicals only on the one with millions. If you have any suggestions of what I could do to assist in debugging let me know as I doubt the postgres log is especially helpful.

2017-12-18 19:41:23 GMT LOG: server process (PID 8196) was terminated by exception 0xC0000005

2017-12-18 19:41:23 GMT DETAIL: Failed process was running: select * from bingo.chemdata where smiles @ ('CCCCCCCN','')::bingo.exact

2017-12-18 19:41:23 GMT HINT: See C include file "ntstatus.h" for a description of the hexadecimal value.

2017-12-18 19:41:23 GMT LOG: terminating any other active server processes

2017-12-18 19:41:23 GMT WARNING: terminating connection because of crash of another server process

2017-12-18 19:41:23 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

2017-12-18 19:41:23 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.

2017-12-18 19:41:23 GMT WARNING: terminating connection because of crash of another server process

2017-12-18 19:41:23 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

2017-12-18 19:41:23 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.

2017-12-18 19:41:23 GMT WARNING: terminating connection because of crash of another server process

2017-12-18 19:41:23 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

2017-12-18 19:41:23 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.

2017-12-18 19:41:23 GMT WARNING: terminating connection because of crash of another server process

2017-12-18 19:41:23 GMT DETAIL: The postmaster has commanded this server process to roll back the current transaction and exit, because another server process exited abnormally and possibly corrupted shared memory.

2017-12-18 19:41:23 GMT HINT: In a moment you should be able to reconnect to the database and repeat your command.

2017-12-18 19:41:23 GMT LOG: all server processes terminated; reinitializing

2017-12-18 19:41:23 GMT LOG: database system was interrupted; last known up at 2017-12-18 19:41:11 GMT

2017-12-18 19:41:24 GMT FATAL: the database system is in recovery mode

2017-12-18 19:41:24 GMT FATAL: the database system is in recovery mode

2017-12-18 19:41:24 GMT LOG: database system was not properly shut down; automatic recovery in progress

2017-12-18 19:41:24 GMT LOG: invalid record length at 49C/88E0F148: wanted 24, got 0

[after this point the server recovers]

AlexanderSavelyev commented 6 years ago

@dan2097 could you please try again with a latest version taken from master branch?

dan2097 commented 6 years ago

@AlexanderSavelyev I've just checked and unfortunately the error is still reproducible with the PgSQL libs built from the latest master. I do have one more insight though: select from bingo.chemdata where smiles @ ('CCCCCCCN','')::bingo.exact limit 1 DOES NOT produce the issue while select from bingo.chemdata where smiles @ ('CCCCCCCN','')::bingo.exact limit 2 does produce the issue The output from both selects are identical, only 1 row is selected. (higher limits also produce the issue)

AliZainoul commented 3 years ago

Please ignore this message: @AliZainoul AliZainoul mentioned this issue 12 minutes ago Can not install python utilities #170 It was a mistake, thank you.

Pandaaaa906 commented 7 months ago

i have the same issue in pg12 with bingo 1.9.1.587-g0e3666ae7