[Feature Request] Highlighting the latest logs within Visualization/Discover

[m0han22]

Feature to Highlight the latest Logs:

We are in need of a feature within Kibana that allows for the highlighting of the latest logs within the Discover or Visualization sections. This feature would enable users to easily identify and focus on the most recent log entries, aiding in real-time monitoring and analysis.

Use Case This feature would greatly benefit individuals like Security Operations Center (SoC) analysts who rely on prompt detection and response to security threats. By highlighting the latest logs, analysts can quickly pinpoint recent activity, identify potential security incidents, and take appropriate actions in a timely manner. This streamlined process not only enhances our overall security posture but also improves operational efficiency.

[elasticmachine]

Pinging @elastic/kibana-data-discovery (Team:DataDiscovery)

[kertal]

@saimohan2k thx for you feedback, could you elaborate what you mean with latest logs? the logs that were added after you opened a data view in Discover when you use Auto-Refresh? The logs of a pre configured timeframe? thx

[m0han22]

@saimohan2k thx for you feedback, could you elaborate what you mean with latest logs? the logs that were added after you opened a data view in Discover when you use Auto-Refresh? The logs of a pre configured timeframe? thx

Hi, @kertal, I mean the logs that were added after we opened a data view in Discover when we use auto-refresh. Basically, our client wants to identify the latest logs in the discovery.

[kertal]

@saimohan2k Thanks the clarification! @stratoula @markov00 I think I've heard a similar request before, but I can't find the issue currently. I don't think we can do this on Lens level, something like showing what's has been added in the histogram compared to an initial state, right?

What I think is what we could do in theory in Discover, having an indication, a background color, an icon, showing when there are new entries/changes in the data table?

[stratoula]

Interesting and yes in the Discover table it would be really cool to identify the newer entries. About the visualizations I am not sure, I want Marco's and @gvnmagni to let me know what they think about it. It seems a bit weird to me. If you have buckets of 10 minutes and the refresh is every 5sec for example? How does it look?

[gvnmagni]

interesting and wide topic here, two notes:

1. to highlights new entries would be great to have something not permanent otherwise it would affect the chart forever potentially causing readability issues. I would love to experiment with a blink, a sort of visual clue that would explain to the user for a brief couple of seconds that those elements are new. Think about having the buckets that have been added to blink and then to keep back being the same color of the rest of the chart.
2. I know we have performances to keep in mind, but it would be great also to consider what happen when we have a refresh, we currently have a complete refresh but what if we introduce animation? Once we have new data the chart automatically shift to the left introducing the new time buckets with a smooth animation instead of a complete refresh. That would be lovely to see, maybe in addition to the blink that I was describing

[kertal]

A thought in our sync by @jughosta : showing when the result changed because of filtering, so new rows were added because of filtering or other changes

[kertal]

another thought in our sync by @davismcphee , showing when the last auto refresh happened, currently there is no way to find our when the last auto refresh was triggered