Open hbgstc123 opened 1 year ago
@hbgstc123 Thanks for raising the issue. @danny0405 could you provide help here?
@hbgstc123 does this happen every few hours or it only happened once so far? can you try upgrading to 0.12.2 and see how it goes?
One suggestion is not to use the session cluster, the session cluster mode is fragile for classloader.
@hbgstc123 does this happen every few hours or it only happened once so far? can you try upgrading to 0.12.2 and see how it goes?
happen every few hours, but after we set this config classloader.check-leaked-classloader = false, it stop to happen
One suggestion is not to use the session cluster, the session cluster mode is fragile for classloader.
we are using application mode
@hbgstc123 does this happen every few hours or it only happened once so far? can you try upgrading to 0.12.2 and see how it goes?
happen every few hours, but after we set this config classloader.check-leaked-classloader = false, it stop to happen
Thanks, seems there are some leak for classloaders, did you use the mor table with async compaction enabled ?
you can set classloader.check-leaked-classloader: "false"
in flink.conf
Describe the problem you faced
flink job, stream read from hudi srouce and stream write to hudi sink. this error happen after run 4 hours, cause job to restart.
java.lang.IllegalStateException: Trying to access closed classloader. Please check if you store classloaders directly or indirectly in static fields. If the stacktrace suggests that the leak occurs in a third party library and cannot be fixed immediately, you can disable this check with the configuration 'classloader.check-leaked-classloader'.
To Reproduce
Steps to reproduce the behavior:
Environment Description
Hudi version : 0.12.1
Flink version : 1.15
Running on Docker? (yes/no) : no
Additional context
Add any other context about the problem here.
Stacktrace