[Serverless]: Discover not working properly for sample data (older dates)

georgewallace commented 3 weeks ago

Serverless Docs

Welcome to Elastic Serverless

Description

The following takes you through my testing of trying to get my example to work

When testing the Aggregations page, I was running the examples using the sample data I used in Get started. This data can be seen in the Resources and additional context section.

In our getting started example we just create an index and add data to the index using the _bulk endpoint. This sets our release_date as our date field. I tried to run the example replacing my-field with page_count' andmy-indexwith books. I got no results. This was because my timestamp fieldrelease_date has old dates from when the books came out.

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      }
    }
  }
}

Next I decided to discover my data as I wanted to look at it. I got the following view.

Knowing that my data is older than 2015, I tried to use the date filter up top to find my records. The absolute picker only gave me dates back to 2008.

I went back to the main page and clicked Search entire time range

This now gives me 1925-1939 as options in the Absolute picker.

If I switch to the relative picker, any range that I pick under 25 years gives me an error.

This experience is confusing to a customer trying to discover their data.

Resources and additional context

{ "index" : { "_index" : "books" } }
{"name": "Snow Crash", "author": "Neal Stephenson", "release_date": "1992-06-01", "page_count": 470}
{ "index" : { "_index" : "books" } }
{"name": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
{ "index" : { "_index" : "books" } }
{"name": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
{ "index" : { "_index" : "books" } }
{"name": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
{ "index" : { "_index" : "books" } }
{"name": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
{ "index" : { "_index" : "books" } }
{"name": "The Handmaids Tale", "author": "Margaret Atwood", "release_date": "1985-06-01", "page_count": 311}

georgewallace commented 3 weeks ago

@KOTungseth This is the example I ran across with the date filters.

elasticmachine commented 2 weeks ago

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

KOTungseth commented 2 weeks ago

Transfered to the Discover team. However, this issue might also occur in Dashboard and Visualizations.

jughosta commented 2 weeks ago

So the problem is that the time range picker does not allow to go back to 1932 for example, right? Yeah, agree, it might be confusing for users.

The @elastic/kibana-visualizations team owns this code. Or it might be the default behaviour from Eui.

kertal commented 1 week ago

I checked in the EUI docs and here it works but also in a quick check running locally

kertal commented 1 week ago

I tried to reproduce but couldn't. Having another look at your description I noticed:

you are searching from 24 years ago (2 weeks ago 24 years ago was October 2000) So you were searching From Oct 2000 to Mar 14th 2000, that's why it's not valid. The first date is nearer to now than the second date. If you choose a different date, e.g. in the year 2001, it should work in this case

davismcphee commented 1 week ago

I think there are couple of different things going on in this issue:

The year dropdown in the Absolute date picker tab only shows +/- 7 from the currently selected year. This is reproducible in the EUI docs, and from a quick scan of the docs, it's not clear if we can currently customize this.
Using a relative date as the start of a range can produce an invalid date range. This is because the selected value is always relative to today's date. This is technically correct, but maybe there's a better way of handling/surfacing it within the EUI date picker.

markov00 commented 1 week ago

There are definitely UX problem with the date picker and we are aware of those. I've checked locally on main but I can't reproduce the same selected date when clicking the Select entire timerange button. So I believe this can be just marked as an issue with the date picker and move the ownership of this to vis team.

kertal commented 1 week ago

I don't think there's a technical issue here, I think it's an UX issue. The displayed error is correct, technically:

elastic / kibana