opensearch-project / dashboards-reporting

Apache License 2.0
9 stars 29 forks source link

[BUG] CSV with filter on `nested` type fields is empty #375

Open joshuali925 opened 1 month ago

joshuali925 commented 1 month ago

What is the bug? CSV with filter on nested type fields is empty

How can one reproduce the bug? Steps to reproduce the behavior:

  1. Create index with a nested field
    
    PUT /test_index
    PUT /test_index/_mapping
    {
    "properties": {
    "tags": {
      "type": "nested",
      "properties": {
        "value": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    }
    }
    }

POST /test_index/_doc { "tags": { "value": "123" } }

2. create index pattern for it
3. go to discover, add filter on `tag.value` is `123`
4. save the search
5. create csv report
6. see it is empty

**What is the expected behavior?**
csv should have results

**What is your host/environment?**
 - OS: [e.g. iOS]
 - Version [e.g. 22] main
 - Plugins

**Do you have any screenshots?**
If applicable, add screenshots to help explain your problem.

**Do you have any additional context?**
This happens because when discover adds the filter on `tag.value`, it is stored as 
```json
{
  "query": {
    "match_phrase": {
      "tags.value": "123"
    }
  }
}

But it is not correct syntax for a nested field. When discover sends the query, it somehow converts this stored filter into correct nested syntax, so discover search still works

{
  "nested": {
    "path": "tags",
    "query": {
      "match_phrase": {
        "tags.value": "123"
      }
    }
  }
}

Reporting doesn't do this step, it uses the first version directly, leading to no results.

A workaround would be to edit the filter as DSL then paste the correct syntax instead of using UI to add this filter.

dblock commented 1 month ago

[Catch All Triage - 1, 2, 3]