search

atomikos / transactions-essentials

Development repository for next major release of
https://www.atomikos.com/Main/TransactionsEssentials
Other
462 stars 139 forks source link

could not flush state image Log corrupted - restart JVM #100

Open chenzhiyuan0402 opened 4 years ago

chenzhiyuan0402 commented 4 years ago

We use spring-boot 2.0.3 and atomikos 4.0.6 in our project.

2019-11-15 09:09:06,329 [WebContainer : 7197dfc6c93a4953a1bfe48b08d71626-f8ffdbd48--4912-T:2ae0d2f6-e9ae-4817-8ea3-c86a53fddf61] WARN CoordinatorStateHandler:40 - Error in committing: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException - recovery will clean up in the background java.lang.IllegalStateException: could not flush state image Log corrupted - restart JVM com.atomikos.recovery.LogReadException at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:54) ~[transactions-4.0.6.jar!/:?] at com.atomikos.finitestates.FSMImp.notifyListeners(FSMImp.java:164) ~[transactions-4.0.6.jar!/:?] at com.atomikos.finitestates.FSMImp.setState(FSMImp.java:251) ~[transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CoordinatorImp.setState(CoordinatorImp.java:312) ~[transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CoordinatorStateHandler.commitFromWithinCallback(CoordinatorStateHandler.java:346) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.ActiveStateHandler$6.doCommit(ActiveStateHandler.java:268) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CoordinatorStateHandler.commitWithAfterCompletionNotification(CoordinatorStateHandler.java:581) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.ActiveStateHandler.commit(ActiveStateHandler.java:263) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CoordinatorImp.commit(CoordinatorImp.java:548) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CoordinatorImp.terminate(CoordinatorImp.java:685) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.imp.CompositeTransactionImp.commit(CompositeTransactionImp.java:282) [transactions-4.0.6.jar!/:?] at com.atomikos.icatch.jta.TransactionImp.commit(TransactionImp.java:172) [transactions-jta-4.0.6.jar!/:?] at com.atomikos.icatch.jta.TransactionManagerImp.commit(TransactionManagerImp.java:414) [transactions-jta-4.0.6.jar!/:?] at com.atomikos.icatch.jta.UserTransactionImp.commit(UserTransactionImp.java:86) [transactions-jta-4.0.6.jar!/:?] at org.springframework.transaction.jta.JtaTransactionManager.doCommit(JtaTransactionManager.java:1034) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:746) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:714) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:532) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:304) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) [spring-tx-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at com.huawei.it.jalor5.security.service.impl.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:100) [jalor-authorization-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at com.huawei.it.jalor5.security.service.impl.internal.SecurityInterceptor.invoke(SecurityInterceptor.java:100) [jalor-authorization-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:688) [spring-aop-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at com.huawei.it.hr.bonus.reimbursement.service.impl.PersonalReimbursementService$$EnhancerBySpringCGLIB$$8ddb519b.queryPersonalReimbursement() [classes!/:?] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_201] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_201] at org.apache.cxf.service.invoker.AbstractInvoker.performInvocation(AbstractInvoker.java:179) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:96) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:192) [cxf-rt-frontend-jaxrs-3.2.5.jar!/:3.2.5] at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:103) [cxf-rt-frontend-jaxrs-3.2.5.jar!/:3.2.5] at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:308) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:121) [cxf-core-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:267) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.ServletController.invokeDestination(ServletController.java:234) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:208) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.ServletController.invoke(ServletController.java:160) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.CXFNonSpringServlet.invoke(CXFNonSpringServlet.java:216) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.handleRequest(AbstractHTTPServlet.java:301) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.doGet(AbstractHTTPServlet.java:225) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at javax.servlet.http.HttpServlet.service(HttpServlet.java:635) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.cxf.transport.servlet.AbstractHTTPServlet.service(AbstractHTTPServlet.java:276) [cxf-rt-transports-http-3.2.5.jar!/:3.2.5] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) [tomcat-embed-websocket-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.hr.bonus.filters.BonusUserFilter.doFilter(BonusUserFilter.java:60) [classes!/:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.jalor5.web.support.filter.AdditionalFilter.doFilter(AdditionalFilter.java:59) [jalor-web-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.celon.bpm.filter.BpmRequestContextFilter.doFilter(BpmRequestContextFilter.java:56) [bpm-external-sdk-jalor-0.0.1.beta.RELEASE.jar!/:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.sso.filter.SsoFilterPub.doFilter(SsoFilterPub.java:175) [sso-login-axis-hae-0.1.jar!/:?] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.jalor5.security.filter.JwtRequestFilter.doFilter(JwtRequestFilter.java:278) [jalor-auth-jwt-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.jalor5.web.support.filter.RequestContextFilter.doFilter(RequestContextFilter.java:73) [jalor-web-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.jalor5.web.support.filter.CharaterEncodingFilter.doFilter(CharaterEncodingFilter.java:29) [jalor-web-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:99) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.springframework.web.filter.HttpPutFormContentFilter.doFilterInternal(HttpPutFormContentFilter.java:109) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.springframework.web.filter.HiddenHttpMethodFilter.doFilterInternal(HiddenHttpMethodFilter.java:93) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:200) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-5.0.7.RELEASE.jar!/:5.0.7.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at com.huawei.it.usf.tracer.handler.server.TracerServerFilter.doFilter(TracerServerFilter.java:70) [jalor-tracer-6.5.3.RELEASE.jar!/:6.5.3.RELEASE] at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:198) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:496) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:140) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:342) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:803) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:790) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1468) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_201] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_201] at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) [tomcat-embed-core-8.5.31.jar!/:8.5.31] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_201] Caused by: com.atomikos.recovery.LogReadException: Log corrupted - restart JVM at com.atomikos.recovery.imp.CachedRepository.assertNotCorrupted(CachedRepository.java:138) ~[transactions-4.0.6.jar!/:?] at com.atomikos.recovery.imp.CachedRepository.get(CachedRepository.java:132) ~[transactions-4.0.6.jar!/:?] at com.atomikos.recovery.imp.OltpLogImp.assertEntryIsAllowedInCurrentState(OltpLogImp.java:35) ~[transactions-4.0.6.jar!/:?] at com.atomikos.recovery.imp.OltpLogImp.write(OltpLogImp.java:30) ~[transactions-4.0.6.jar!/:?] at com.atomikos.persistence.imp.StateRecoveryManagerImp.preEnter(StateRecoveryManagerImp.java:51) ~[transactions-4.0.6.jar!/:?] ... 110 more 2019-11-15 09:09:06,332 [WebContainer : 7197dfc6c93a4953a1bfe48b08d71626-f8ffdbd48-y00254986-4912-T:2ae0d2f6-e9ae-4817-8ea3-c86a53fddf61] DEBUG ExceptionHandler:179 - No converter found.Jalor could not convert exception:org.springframework.transaction.UnexpectedRollbackException 2019-11-15 09:09:06,333 [WebContainer : 7197dfc6c93a4953a1bfe48b08d71626-f8ffdbd48-y00254986-4912-T:2ae0d2f6-e9ae-4817-8ea3-c86a53fddf61] ERROR ExceptionHandler:195 - Code:[null],User:[y00254986],[Error:JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is javax.transaction.RollbackException: Error in committing: could not flush state image Log corrupted - restart JVM

GuyPardon commented 4 years ago

This is strange. Did you try with 5.0.3?

chenzhiyuan0402 commented 4 years ago

This is strange. Did you try with 5.0.3?

Thank you for your reply. We'll try 5.0.3, but we don't know in which condition this issue will happen.

GuyPardon commented 4 years ago

Me neither, it's not a common issue. You mean it does not repeat then?

chenzhiyuan0402 commented 4 years ago

Yes, not repeat now. Our project is running for several months, this issue happened for first time.

GuyPardon commented 4 years ago

OK it probably means some transient / temporary disk IO failure then.

chenzhiyuan0402 commented 4 years ago

We have two test environment, and two web server, they have different disks. But the issue happened in two server almost at the same time.

jenopob commented 4 years ago

the same question as #98,I tried 5.0.3 ,the problem not solved yet!any suggestion?

NarimanAB commented 4 years ago

This is how it is possible to reproduce the problem in Spring Boot env:

  1. Begin new transaction (for example with @Transactional annotation)
  2. Inside that transaction start new transaction in a new thread (for example: make new CompletableFuture with a long running code inside new transaction with the help TransactionTemplate, submit it to an executor service and keep the reference to the Future)
  3. On the initial thread, while CompletableFuture is still running call "future.cancel(true)". Why do we need to call "future.cancel(true)"? One of the possibility is that we waited for the Future to complete in X seconds but it did not and we would like to recover (at least to try).
  4. At this step, Atomikos goes to endless recovery attempts and keeps asking to "restart JVM" via logging: "could not flush state image Log corrupted - restart JVM"

Maybe step 1 is not necessary.

I agree that calling "future.cancel(true)" is a bit "cruel" but... why it should completely break consistency of JTA log. By the way, in JavaEE (JBoss 7.1 with its default JTA manager) it works just fine and transaction is rollbacked after "future.cancel(true)" is called.

GuyPardon commented 4 years ago

Thanks, if you can supply a maven sample project then we can have a look.

NarimanAB commented 4 years ago

This is the maven sample project to reproduce the issue: https://github.com/NarimanAB/atomikos.git just run the test to see the messages about corrupted JTA logs.

I think the problem is that in my test I catch InterruptedException and restore the thread state. I do it to emulate long running, not responding processes (as an example a long running DB operation). Without interrupted state restore Atomikos correctly handles exception, but with a thread interrupted state restored the whole JTA log becomes corrupted.

GuyPardon commented 4 years ago

Thanks - we'll have a look

kschroiff commented 3 years ago

FWIW, I can see "Log corrupted - restart JVM" as well

Atomikos 5.0.8 / Spring Boot 2.4.2 (on Kubernetes with 3 pods in a specific service)

Seems to be some kind of cyclic cleanup job that is running there - the message shows up every 120secs - which corresponds to the set transaction time-out ( com.atomikos.icatch.default_jta_timeout=120000 ). When reducing the timeout to 60secs, the log has an interval of 60secs as well.

java.lang.Thread.run(Thread.java:834) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128 com.atomikos.timing.PooledAlarmTimer.run(PooledAlarmTimer.java:88 com.atomikos.timing.PooledAlarmTimer.notifyListeners(PooledAlarmTimer.java:101 com.atomikos.icatch.imp.RecoveryDomainService$1.alarm(RecoveryDomainService.java:55) com.atomikos.icatch.imp.RecoveryDomainService.performRecovery(RecoveryDomainService.java:76) com.atomikos.recovery.fs.RecoveryLogImp.getExpiredPendingCommittingTransactionRecordsAt(RecoveryLogImp.java:52) com.atomikos.recovery.fs.CachedRepository.findAllCommittingCoordinatorLogEntries(CachedRepository.java:145) com.atomikos.recovery.fs.CachedRepository.assertNotCorrupted(CachedRepository.java:137) Log corrupted - restart JVM: com.atomikos.recovery.LogReadException: Log corrupted - restart JVM

GuyPardon commented 3 years ago

Thanks for the feedback @kschroiff - are you also doing threading and canceling their work?

kschroiff commented 3 years ago

@GuyPardon In the meantime, I found the issue - see:

https://github.com/atomikos/transactions-essentials/issues/139

Even so, it feels strange that this results in a corrupted log.

GuyPardon commented 3 years ago

We merely assume the log is corrupted. After checking the code I think we can be more optimistic - with a few minor changes. I used @NarimanAB 's test suite to diagnose.

hopefullyhh commented 3 years ago

I meet a similar problem with yours as well @kschroiff

ChenJiahui0 commented 2 years ago

I meet a similar problem with yours as well

CloudShen-107 commented 9 months ago

I meet the same problem when I use transactionTemplate.execute(...). Whenever an exception ocurrs in execute function, the issue well happen, and it keeps retrying. My version is springboot 2.7.18 and atomikos 4.0.6.