Retry problem with deferred write [BATCH-1873]

[spring-projects-issues]

Giovanni Dall'Oglio Risso opened BATCH-1873 and commented

We found a problem in a chunk with retry enabled.

We use MyBatis as writer

• in older MyBatis versions, the "add batch feature" was not supported
• so, we used "for each item / write single item" approach

We had production problems:

• if an exception is thrown during the commit phase
  • all the chunk is lost
  • SB continues with the other chunks as nothing happened

We build a test case showing the problem (please: find attached).

Background: we have a table with not-nullable columns, and we have a job, with a chunk that

• Reader: generates 500 values
• Writer: at some moments, inject a null-value in the items-list to be writter (causing the error), then delegates the real write

In this test case, we tested 4 scenarios (= 4 delegates)

• JdbcBatchItemWriter -> works fine
• SingleUpdateItemWriter (the "for each item / write single item" approach, using JdbcTemplate) -> works fine
• MyBatisBatchItemWriter (the JdbcBatchItemWriter for MyBatis, provided by MyBatis guys) -> works fine
• MybatisItemWriter (the "for each item / write single item" using MyBatis) -> the broken one

It seems to me, that the three working scenarios shares the "write immediatly" approach: the error is detected while in write(List l) method, so the retry template is able to intercept the error, and manage it.

The failing one, defer the real write at the TX.commit() time, that is too late for being managed.

To better explain the idea, i attached the screenshot of my stack, i see:

• Repeat (green)
  • TX (red) * Retry** (blue)

The first-level repeat explains why SB continues with the other chunks.

A simple solution could be to have a "simulated restart of a failed job", so the itemstreams reposition the elaboration, and everything can be continued, but this way you loose the optimizations (the internal readed-items list).

In our case, we work-arounded the problem adopting the MyBatisBatchItemWriter, but i'm writing this issue because you can choose to investigate this problem: deferred write could be a common scenario (I remember something like that in the FlatFileItemWriter, that is ways more impacting)

---

Affects: 2.1.8

Attachments:

• SB-retry problem stack.jpg (143.95 kB)
• SB-Retry-Test.zip (16.69 kB)

[spring-projects-issues]

Giovanni Dall'Oglio Risso commented

I'm sorry, I attached thwice the screenshot. Can somebody remove the duplication? Thanks

[spring-projects-issues]

Michael Minella commented

Are you using iBatis or MyBatis? We don't support any version of MyBatis since it switched. If you want to use MyBatis (post fork), you should be using the ItemWriter they provide.