This contains 3 PRs worth of work and will be refactored.
The sequence is as follows:
While race-testing the new integration test, the Arrow receiver turned up a bug identified by the race detector. the inFlightWG was shared by all streams in the receiver, so it would work in testing (almost always) but was incorrect. This may explain unusually long stream shutdowns.
The Arrow exporter was not consistently using gRPC status errors, which the Reciever had fixed in https://github.com/open-telemetry/otel-arrow/pull/205; the code was out-of-line with the design of the top-level directory. Whereas the OTel-Arrow exporter had been inserting consumererror.NewPermanent() wrappers, it is the Exporter module which supports standard OTLP and Arrow one layer up that is responsible for permanent error labeling. Returning gRPC status errors is always preferred to fmt.Errorf in gRPC components.
A new memory-limited integration test is added, which exposed the need for correct status error handling, also exposed an Arrow Consumer bug. The underlying Arrow IPC Reader object has an Err() method that was not being checked--instead we had (3a) a fallback mechanism, checking expected record count, and (3b) write to os.Stderr because of the (known) missing error.
This contains 3 PRs worth of work and will be refactored. The sequence is as follows:
inFlightWG
was shared by all streams in the receiver, so it would work in testing (almost always) but was incorrect. This may explain unusually long stream shutdowns.consumererror.NewPermanent()
wrappers, it is the Exporter module which supports standard OTLP and Arrow one layer up that is responsible for permanent error labeling. Returning gRPC status errors is always preferred tofmt.Errorf
in gRPC components.Err()
method that was not being checked--instead we had (3a) a fallback mechanism, checking expected record count, and (3b) write to os.Stderr because of the (known) missing error.