Closed jmacd closed 3 months ago
@kristinapathak Proposing to add you as a reviewer in this repository. This change is a substantial rewrite but not a substantial change of behavior, so it's worth a careful review and should help you learn the receiver code very well. Thank you! 😁 🚀
Reviewers, please take another look.
For the test flake above, https://github.com/open-telemetry/otel-arrow/issues/206
Restructure receiver code to improve readability. There are a number of metrics that are incremented when a batch starts being processed and are decremented when the batch is finished, but the control flow that maintained the balance of these updates was convoluted.
The root-cause of #204 is that Arrow batches meant for a consumer to be processed in order were processed out-of-order. There was a large function body which served two purposes: consume Arrow data of the appropriate kind, enter data for the pipeline to consume next. This had to be split into two parts and should have been done as part of #181. (I, as reviewer, missed this and find, in hindsight, that the code is not easy to follow.)
This improves the code structure by moving all stateful aspects of starting/finishing a request into a new
inFlightData
object which has a deferrable method to finish the request. Here, we keep:inFlightWG
done countAuthorization now happens before acquiring from the semaphore.
A number of
fmt.Errorf()
calls are replaced withstatus.Errorf(...)
and a specific error code. The tests are updated to be more specific. Several Arrow tests were accidentally canceling the test before an expected error condition was actually tested, they have been audited and improved.One new concurrent-receiver test was added.
Fixes #204.