The hardcoded number of partitions in conjunction with limiting the number of executors to significantly less than number of partitions leads to a problem on x100 side. If the filled data stream output queues are not fetched then a point is reached, where a partition queue is completely filled up. In this situation, no more input tuples can be processed and hence we have a deadlock situation.
The fixe tries to mitigate the problem by setting the number of partitions to the "expected" number of executors for the job.
The hardcoded number of partitions in conjunction with limiting the number of executors to significantly less than number of partitions leads to a problem on x100 side. If the filled data stream output queues are not fetched then a point is reached, where a partition queue is completely filled up. In this situation, no more input tuples can be processed and hence we have a deadlock situation.
The fixe tries to mitigate the problem by setting the number of partitions to the "expected" number of executors for the job.