Open tvvister opened 2 months ago
This is an expected behavior. They are not swallowed. They are saved and available to you if you call future.result()
or future.exception()
. If you want to log errors even if they are ignored by your code, you should wrap the body of your tasks with try
/except
. You can do this with a wrapper function or a decorator.
Changing this is a breaking change. Logging every raised exception will break programs which correctly handle them.
I suggest to log only raised exceptions, which are not handled at all. I guess it means that failed future (or task) not await
-ed and neither method exception
nor result
were called for corresponding future, awaitable or task (maybe something else like this).
This would be devastating too. You can stop at the first error and do not check other futures. Ignoring some exceptions can be normal behavior, for example when you you finish the work by closing a file and leaving tasks to die silently from OSError.
Write a simple wrapper:
def print_error(func):
def wrapper(*args, **kwargs):
try:
func(*args, **kwargs)
except BaseException as e:
print(f'Error in {e!r} in {func}')
raise
and wrap every function passed to pool.submit()
.
Sure, I will use the idea, you express in the previous msg.
But I cannot still accept internally the design. I assume it will take a little bit more time.🙂 I get used to the design which applied in .net. Maybe this is the reason of my slowness. In .net you can use one global handler for every exception in a separate thread.
Here is much more detailed:
Bug report
Bug description:
This code just loses an exception. Without any notification.
The reason is in ThreadPoolExecutor's exceptions swallowing. Proposals are:
The problem maybe seems too exaggerated, but if you have a some year old project with a bunch of places like this, it would mean probably you have a lot of hidden errors and need to analize all the code and fix with
try ... except
inside task or callingresult()
for every single future, which you got as a returned value ofThreadPoolExecutor.submit
. It sounds old old fashioned and smells like a bad design.By the way, if we could at least catch all these kind of unhandled exceptions, it would be a good solution. Unfortunatly, neither
sys.excepthook
northreading.excepthook
catch anything interesting.All the same is valid for
ProcessPoolExecuter
as wellCPython versions tested on:
3.11
Operating systems tested on:
Linux