search

Segfault-Inc / Multicorn

Data Access Library
https://multicorn.org/
PostgreSQL License
700 stars 145 forks source link

Catch and handle "canceling statement due to user request" events? #140

Open ergo70 opened 8 years ago

ergo70 commented 8 years ago

Hi,

I've wrote a multicorn FDW that spawns multiple processes using Python's multiprocessing module.

Everything works fine, except that when I cancel a query manually (server says: "canceling statement due to user request"), the parent process stops execution but the children workers continue until they have finished their work.

This is a problem if a user cancels too many long running queries, since the workers don't get killed and choke the system.

I have the code to kill them but how do I know, when the FDW cancels a query so that I can kill the workers? Is there some hook or signal that I can use to detect 'canceling statement due to user request' events? end_scan() apparently is not called when a query is cancelled...?

best regards,

Ernst-Georg

ergo70 commented 8 years ago

I can confirm that end_scan() is called when execute() finishes naturally, but not when the query is cancelled by the user. How to I catch forcibly cancelled queries?

klaussfreire commented 5 years ago

Maybe we should have an abort_scan(). I'm having similar issues with an SQLAlchemy FDW that queries a Presto server. It leaves Presto queries abandones, which is worse even since those queries will stay there consuming RAM until the timeout runs out, which can be hours, because Presto's API is an HTTP polling API (so it doesn't detect connection resets).

If we had an abort_scan() I might insert a cancel request there to abort the remote side as well. This will probably apply to all SQLAlchemy FDWs in fact.