My suggestions, feel free to add yours. I suggest to spawn new issues for every task and discuss the details there, to keep this meta issues clean.
[ ] point in time breaks on shard relocation, @dnhatn is already looking into this
[ ] search fails with a QueryShardException, this exception type classifies itself as BAD_REQUEST, but not every QueryShardException is a BAD_REQUEST. A bad request is considered an irrecoverable error, that's why transform decides not to retry(except it is configured as unattended)
[ ] the underlying Lucene exception of type AlreadyClosedException is a subtype of IllegalStateException, this is more of a 409, if the exception wouldn't have been turned into QueryShardException, it would have been treated as internal server error. In both cases transform would have retried.
[ ] The Lucene exception lacks useful information, e.g. the name of the analyzer (not sure if this is possible)
[ ] SearchExecutionContext should add important information to the exception, if possible, e.g. name of the field, the sub-query, ...
[ ] Transform uses getDetailedMessage() to populate the reason field in _stats to persist the error(_stats is used in the diagnostics bundle). The reason field lacks useful information. Here are some requirements, proper naming, API design, UI implications, to be discussed as separate issue:
[ ] transform should log/persist the last executed query in order to help debugging, this could be added as additional field (only visible via _stats, no UI)
[ ] transform should use the toXContent(...) method of ElasticsearchException together with proper params to populate another field with the full JSON output of this method. In contrast to getDetailedMessage(), this includes the stack trace
Based on recent feedback we discovered some shortcomings in transforms when search experiences a problem. This meta issue lists improvements.
This is not a transform issue, but covers search, which builds the foundation for transform.
/CC @dnhatn, @javanna, @przemekwitek, @wchaparro, @droberts195, @sophiec20, @dimkots
My suggestions, feel free to add yours. I suggest to spawn new issues for every task and discuss the details there, to keep this meta issues clean.
QueryShardException
, this exception type classifies itself asBAD_REQUEST
, but not everyQueryShardException
is aBAD_REQUEST
. A bad request is considered an irrecoverable error, that's why transform decides not to retry(except it is configured asunattended
)SearchExecutionContext
treats a genericException
asQueryShardException
AlreadyClosedException
is a subtype ofIllegalStateException
, this is more of a 409, if the exception wouldn't have been turned intoQueryShardException
, it would have been treated as internal server error. In both cases transform would have retried.SearchExecutionContext
should add important information to the exception, if possible, e.g. name of the field, the sub-query, ...getDetailedMessage()
to populate thereason
field in_stats
to persist the error(_stats
is used in the diagnostics bundle). Thereason
field lacks useful information. Here are some requirements, proper naming, API design, UI implications, to be discussed as separate issue:_stats
, no UI)toXContent(...)
method ofElasticsearchException
together with proper params to populate another field with the full JSON output of this method. In contrast togetDetailedMessage()
, this includes the stack trace