Remove the break identifier from DateFieldFormatter, the current behaviour will only parse the first date field from the resultSet, which is not a valid assumption, because OpenSearch as platform allow multiple date fields co-exist on an index, also this is not an uncommon use-case for enterprise. Hence all date fields present on any given index should be formatted regardless of the order.
To detect and attempt parse any unix timestamp (second || milisecond) on best-effort basis, OpenSearch as platform allow unix timestamp value to be inserted despite epoch_millis being explicitly specified on field format, in this particular case there is no clear hint from the query indicate this is numeric timestamp from OpenSearch.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.
Description
This is a MR, which aim to fix the issue described over https://github.com/opensearch-project/sql/issues/1545 to align the representation of
date
field between the legacy engine and the v2 engine.Code changes highlight:
break
identifier fromDateFieldFormatter
, the current behaviour will only parse the firstdate
field from the resultSet, which is not a valid assumption, because OpenSearch as platform allow multipledate
fields co-exist on an index, also this is not an uncommon use-case for enterprise. Hence alldate
fields present on any given index should be formatted regardless of the order.epoch_millis
being explicitly specified on field format, in this particular case there is no clear hint from the query indicate this is numeric timestamp from OpenSearch.Although issue is being reported https://github.com/opensearch-project/sql/issues/1545 under nested-query, however this bug has wider impact outside of nested-query, and the conditions to trigger are:
date
fields are being specified on the Index schema WITHOUT any explicitformat
configuration.When this happen, the current behaviour will only parse the first
date
field from the resultset.Reference doc:
Related Issues
Resolves https://github.com/opensearch-project/sql/issues/1545
Testing plan
Check List
--signoff
.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license. For more information on following Developer Certificate of Origin and signing off your commits, please check here.