[BUG] Visalization Controls does not show data

[bebablub]

Describe the bug

We are using OpenSearch and OpenSearch Dashboards in AWS with version OpenSearch_2_11_R20231113-P2 (latest)

We indexed data in the following format (example): { "_index": "mme_feedback_exception", "_id": "f3e63203e83a4e4ba722c29af5b504bb/agg_feedback.txt_1703947876000", "_version": 1, "_score": null, "_source": { "timestamp": 1703947876000, "component": "AbstractCommunicator", "message": "ConnectionToVciLostException null: Error reading battery-voltage", "stack_trace": "None", "type": "Connection", "meta": { "marke": "VW", "serie": "Touareg", "modell": "7P (2010 – 2018)", "versions": [ "202310270944" ], "vega": "vega" } }, "fields": { "timestamp": [ "2023-12-30T14:51:16.000Z" ] }, "sort": [ 1703947876000 ] } and have around 500000 documents indexed within a timerange of the last 90 days. In a timerage of 30 days it's still around 25000 documents. Discovery shows that all documents having same structure and all fields indexed. A mapping is set for all fields

We created a controls visalization to have a dropdown field for the field "type". For a timerange of 90 days it shows an tooltip with:

For a timerange of 30 days (so less time range, less data) it shows actual types

To Reproduce

• Index data in the above format
• Create a controls visualization as "Options List"
• Select correct Index Pattern
• Select "type.keyword" as Field
• allow Multiselect, disallow Dynamic Options, choose size of 5
• at Options select "Update OpenSearch Dashboards filters on each change" and " Use time filter"
• Update
• Select Last 90 Days... error tooltip shows

Expected behavior Data is shown also in a time range of 90 Days like with 30 days cause 90 days also includes 30 days

OpenSearch Version OpenSearch_2_11_R20231113-P2 (latest)

Dashboards Version v 2.11.0

Plugins no plugins explicit enabled

Host/Environment (please complete the following information): AWS t3.medium.search EC2 type Less then 5GB of indexed data

Additional Infos Searched already for that issue and found some entries which could be intersting

Exact the same issue: https://github.com/elastic/kibana/issues/31804 But as type is not a nested field this is obviously not the issue... unfortunately it's closed

Same error: https://discuss.elastic.co/t/empty-drop-down-list-filtering-occurs-on-field-which-does-not-exist/190898 But our documents are all same structured and we checked "Use time filter" which made 30 days to work (otherwise the controls viszalization would not work at all for any timerange). So probably a different issue

Goes in the same direction but shows at least something: https://github.com/opensearch-project/OpenSearch-Dashboards/issues/4009 Unfortunately as we are hosted at AWS we cannot modify the opensearch-dashboards.yml

Anyone has an idea? Thank you!

[AMoo-Miki]

Option lists are limited to what they can query; when the control makes the call to get a list of values, it does so with the default size and if within those 500/1000 records, the particular key doesn't have a value, the control would gray itself out and show that warning. While we have a few thoughts on how to improve this control, it being experimental leads to it being not looked at as often.

[wbeckler]

@AMoo-Miki what kind of thoughts are there on how to improve this behavior?