opensearch-project / OpenSearch-Dashboards

📊 Open source visualization dashboards for OpenSearch.
https://opensearch.org/docs/latest/dashboards/index/
Apache License 2.0
1.67k stars 871 forks source link

[BUG] is filter not working with numeric Scripted fields #6996

Closed LDrago27 closed 2 months ago

LDrago27 commented 3 months ago

Describe the bug Adding filters for scripted numeric fields using is operator after clicking the Add Filter Button does not produce any result. However directly clicking on a visualization containing the required field adds the filter correctly. This behavior is observed across visualizations. (Tested the behavior in VisBuilder & Vertical bar Visualization)

Payload sent while using the Add Filter button PayloadAddFilter.json Payload being sent while clicking on the visualization directly PayloadVisualization.json

Upon analysis of Payloads, the following observation was found. When using the Add Filter button, The value is being sent as string. "value": "66" While when we are using the visualization to filter it sends the "value": 66

This explains why we are not getting the expected results with the Add Filter option in case of numeric fields since we are sending the value as string.

Expected behavior Add Filter should work with scripted numeric fields while using the is operator.

OpenSearch Version 3.0.0-snapshot Dashboards Version 3.0.0

Plugins

Please list all plugins currently enabled.

Screenshots

Using the Add filter image

Clicking on the Visualization image

Host/Environment (please complete the following information):

Additional context

Add any other context about the problem here.

abbyhu2000 commented 3 months ago

Was able to reproduce this bug.

When we add a filter by clicking on one of the bar on the visualization, it will send the query value as number type, "value": 66; whhen we add a filter using add filter button from the filter bar, it will send the query value as string type, "value": "66".

The above flows are true for both numeric scripted field, and any numeric fields.

However, when we send a query containing a value of string type for other numeric fields, ex. for opensearch_dashboards_sample_data_ecommerce, when we add filter like products.price: "66", it will still work correctly. But whenever we send a query containing a value of string type for numeric scripted fields, the filter does not work.

abbyhu2000 commented 3 months ago

When we create a scripted field, if we set the type as number, add filter will not work since the filter bar treats the input as a string.

However, if we created the same scripted field as a string type, add filter will work correctly.

Screenshot 2024-06-11 at 3 22 51 PM

So add filter will not work for actual_price, but will work for actual_price_string.

So i think we should probably modify the filter bar to solve this issue? Like after user choose the field, and if the field is numeric type, the value input textbox should convert the string into a number so scripted field can work properly? @kavilla @AMoo-Miki

Screenshot 2024-06-11 at 3 26 19 PM
abbyhu2000 commented 3 months ago

For boolean scripted field, the filter bar detects the type as boolean, and it populates boolean values to send, so the filter bar is working properly for boolean scripted field.

https://github.com/opensearch-project/OpenSearch-Dashboards/assets/43937633/d7cf4307-6877-4add-9be5-fecfc97dacc0