Previously, working under the assumption that a fetch response would
have one record batch with multiple records, we ignored EOF errors while
parsing a record, if we had at least one record already. (Or we tried,
we missed detecting the error at the right place, but this commit also
fixes that).
But since a response can actually contain multiple record batches, it's
also valid to have the final batch cut off in the middle.
To fix this, we hoist the EOF check up one layer and ignore it if we
have at least one successful batch.
There's still a potential issue where a single large batch might not fit
into the response, but we could theoretically make progress by returning
some of the records within the batch that we did get completely.
However, I believe kafka will always send complete batches in a
response, so the next request, even though it moves forward in terms of
the offset that it requests, would still get the same response and make
no new progress.
Previously, working under the assumption that a fetch response would have one record batch with multiple records, we ignored EOF errors while parsing a record, if we had at least one record already. (Or we tried, we missed detecting the error at the right place, but this commit also fixes that).
But since a response can actually contain multiple record batches, it's also valid to have the final batch cut off in the middle.
To fix this, we hoist the EOF check up one layer and ignore it if we have at least one successful batch.
There's still a potential issue where a single large batch might not fit into the response, but we could theoretically make progress by returning some of the records within the batch that we did get completely. However, I believe kafka will always send complete batches in a response, so the next request, even though it moves forward in terms of the offset that it requests, would still get the same response and make no new progress.