Jet can lose stream items on member reconnection close to job completion

[viliam-durina]

Scenario:

• SenderTasklet on member1 sends an item to member2, followed by a DONE_ITEM, and completes
• due to a connection glitch, the item is lost, but not the DONE_ITEM
• ReceiverTasklet on member2 receives the DONE_ITEM. It doesn't detect connection change because there was no flow control packet in the meantime.
• The job successfully completes, but produces wrong results.

It's pretty hard to happen in real life because flow control packets are sent every 100ms, the stream item and the done item have to be in two batches, the member has to reconnect in that short time and all tasklets have to complete in that short window. But, on the other hand, it should be pretty easy to fix - to check the source connection for each packet in ReceiverTasklet. But maybe there's a gotcha...

[Holmistr]

Based on:

It's pretty hard to happen in real life

moving it to backlog for now and we'll consider it during planning for the next releases.