search

embulk / embulk-output-jdbc

MySQL, PostgreSQL, Redshift and generic JDBC output plugins for Embulk
Other
88 stars 86 forks source link

Add logging to avoid loss of information when exception changes during retries #325

Closed joker1007 closed 11 months ago

joker1007 commented 1 year ago

If a raised exception changes during retry executions and the exception is not retryable, the retrying is aborted instantly, but the information of the last exception is lost, because RetryExecutor throws only the first exception and OnRetry hook doesn't log about the last exception.

For example, like this:

2023-09-06 18:56:18.365 +0000 [WARN] (0018:task-0010): Operation fai
led (1213:40001), retrying 1/8 after 1 seconds. Message: Deadlock found when trying to get lock; try restarting transaction
2023-09-06 18:56:19.366 +0000 [ERROR] (0018:task-0010): Operation fa
iled (1213:40001)

In this situation, it is difficult to identify the root problem, so I would like to change to log also last exception.

joker1007 commented 11 months ago

A reason why I didn't use buildExceptionMessage is that the method displays only error messages. I wanted the stack trace of the exception.

I don't think the logging format needs to be aligned with onRetry. Because users usually don't need the stack trace of retriable exceptions but typically want the stack trace of not retriable exceptions or given up exceptions.

dmikurube commented 11 months ago

Got it. Sounds reasonable! Let me merge this.