Describe the feature:
Follow up to https://github.com/JanusGraph/janusgraph/issues/4408. Since implementing that feature I've noticed bulk requests do not get retried. In particular I periodically will have circuit_breaking_exception error messages mentioned by the thrown IOException here.
It appears the bulk API doesn't participate as expected with regards to ResponseException.
It appears it can "successfully" return a Bulk Response, that contains items within it that failed. This PR applies the retry logic originally added in https://github.com/JanusGraph/janusgraph/issues/4408 if all the failures within the bulk response set returned error codes that are configured for retry.
Circuit Breaker Exceptions in particular are coded on the server side as being another type of 429 Too Many Requests that would be appropriate for back pressure to be applied. Operationally some circuit breaker errors (like the parent circuit breaker) are OOM preventative measures to the calling client just saying your request right now may cause an OOM, try again later.
Describe a specific use case for the feature:
This feature would allow bulk request item failures to be reactive to backpressure via error codes.
Describe the feature: Follow up to https://github.com/JanusGraph/janusgraph/issues/4408. Since implementing that feature I've noticed bulk requests do not get retried. In particular I periodically will have
circuit_breaking_exceptionerror messages mentioned by the thrown IOException here.It appears the bulk API doesn't participate as expected with regards to
ResponseException.It appears it can "successfully" return a Bulk Response, that contains items within it that failed. This PR applies the retry logic originally added in https://github.com/JanusGraph/janusgraph/issues/4408 if all the failures within the bulk response set returned error codes that are configured for retry.
Circuit Breaker Exceptions in particular are coded on the server side as being another type of 429 Too Many Requests that would be appropriate for back pressure to be applied. Operationally some circuit breaker errors (like the parent circuit breaker) are OOM preventative measures to the calling client just saying your request right now may cause an OOM, try again later.
Describe a specific use case for the feature:
This feature would allow bulk request item failures to be reactive to backpressure via error codes.