opensearch-project / opensearch-hadoop

Apache License 2.0
29 stars 22 forks source link

Make invalid request exceptions more detailed #438

Closed Xtansia closed 2 months ago

Xtansia commented 3 months ago

Description

Due to the logic here in certain circumstances when a response indicates a client error (4xx status) but the response body is not an error object it may result in a non-sensical exception message like seen here:

org.opensearch.hadoop.rest.OpenSearchHadoopInvalidRequest: null
null

This change makes it so that the exception message always contains detailed information about the request being made and the response body, and if the response was an error it assigns it as the "cause" of the invalid request.

For example if you had your max scroll count setting too low the error would now look like:

Exception in thread "main" org.opensearch.hadoop.rest.OpenSearchHadoopInvalidRequest: [POST] on [test-index/_search] failed; server[127.0.0.1:9200] returned [429|Too Many Requests]
    at org.opensearch.hadoop.rest.RestClient.checkResponse(RestClient.java:489)
    at org.opensearch.hadoop.rest.RestClient.execute(RestClient.java:445)
    at org.opensearch.hadoop.rest.RestClient.execute(RestClient.java:439)
    at org.opensearch.hadoop.rest.RestClient.execute(RestClient.java:419)
    at org.opensearch.hadoop.rest.RestRepository.scroll(RestRepository.java:325)
    ... <SNIP>
Caused by: org.opensearch.hadoop.rest.OpenSearchHadoopRemoteException: rejected_execution_exception: Trying to create too many scroll contexts. Must be less than or equal to: [0]. This limit can be set by changing the [search.max_open_scroll_context] setting.
    at org.opensearch.hadoop.rest.ErrorExtractor.extractErrorWithCause(ErrorExtractor.java:54)
    at org.opensearch.hadoop.rest.ErrorExtractor.extractError(ErrorExtractor.java:92)
    at org.opensearch.hadoop.rest.RestClient.checkResponse(RestClient.java:473)
    ... 39 more

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.