[Security Solution] Single alert may be hidden due to table height

[nicpenning]

Kibana version: 8.8.0 Elasticsearch version: 8.8.0 Server OS version: Windows Server 2019 Browser version: Version 113.0.1774.57 (Official build) (64-bit) Browser OS version: Windows 11 Original install method (e.g. download page, yum, from source, etc.): Downloads page Describe the bug: From time to time we navigate to the alerts page from a link in an email or other medium and the table height is not large enough to display the alert. Steps to reproduce:

1. Navigate to alerts page
2. See that the alert is nearly hidden

Note: I am unsure how to reproduce this behavior.

Expected behavior: We should be able to see the alert every time.

Also, if I refresh the page, it will correct the problem.

Screenshots (if relevant): https://github.com/elastic/kibana/assets/5582679/e2d228f1-9345-4eae-98f4-04cc4b77a566

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

Any additional context: We use the webhook action to send alerts to third party apps such as Teams or use the email action with the alert context.

A sample URL looks like this: https://elastic-url/app/security/detections/rules/id/0e7ed8f6-563b-40b2-a871-ce9ee2f03737?timerange=(global:(linkTo:!(timeline%29,timerange:(from:1686068179567,kind:absolute,to:1686068487233%29%29,timeline:(linkTo:!(global%29,timerange:(from:1686068179567,kind:absolute,to:1686068487233%29%29%29

This specific alert is from the built in Endpoint Security detection rule.

[MakoWish]

Related to #159003? Seems like a regression to #134268

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[MadameSheema]

Thanks for the report @nicpenning!!

Yes, this seems to be related.

[elasticmachine]

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

[elasticmachine]

Pinging @elastic/response-ops (Team:ResponseOps)

[MakoWish]

It actually seems like a new issue. Previously, the table would show a row that is too small to display the data. Now, the table shows proper row heights, but no data is displayed. Here is a screenshot showing three detections, but zero data is displayed.

Also to note... I tried selecting one of the rows, then "Investigate in timeline", and nothing happens. I have no way of viewing the alert details. As an Information Security team, this is a major bug for us.

[angorayc]

https://github.com/elastic/kibana/assets/6295984/0c99c4d1-8662-4264-b6f2-3de6f33ecc36

https://github.com/elastic/kibana/assets/6295984/99383e65-a0f6-43cb-8a6c-b7d1efea20c1

[MakoWish]

Any update on this?

[logeekal]

@MakoWish , We are still looking into it and it is taking time as this particular error is difficult to reproduce.

[spong]

@MakoWish, as a workaround, it seems like these alerts display fine within Timeline (just not the Alerts Table), so you should be able to use that view for the time being till we can get a fix in place.

[MakoWish]

If any of you want to jump on a call to see this replicated, or to grab any other information that may help, we can work that through the ticket I have open. I can replicate this every time on one specific rule.

[kqualters-elastic]

@MakoWish @nicpenning we have finally identified exactly what's causing each of these 2 bugs, and fixes are incoming for both.

First issue, where the alerts table would render with a height of 0, linked in the original comment in this thread and seen sporadically over the last 2 years, ultimately was a bug in our component library eui, where users viewing 1 alert + in some way had modified os or browser settings to "zoom" in or make text larger than it is by default, would cause some math done in the table component to calculate height/width to fail due to comparing a float vs an int, 50 !== 49.999999997. https://github.com/elastic/eui/pull/6895 will be in the next versions of kibana. Workarounds: Adjust the os zoom level settings and refresh the page, or zoom the browser in and out.

I'm able to reproduce it with text size set to 105%-109%, anything outside of that and the table seems to work as expected, but your milage may vary. There are also a whole lot of other ways that I think this could happen, but haven't reproduced them yet, so apologize if what's causing it for you is not mentioned. Browser extensions, custom system fonts are two that come to mind but there are probably many more.

Second issue: This is the one most discussed in this thread, and ultimately stems from a bug in elasticsearch. https://github.com/elastic/elasticsearch/issues/97684 Threat indicator alerts will have a property 'threat.indicator' that is of nested type, which contains a 'file.name' field that we unfortunately include in the request as it's a default column. This combination of requesting all fields via:

            {
                "field": "*",
                "include_unmapped": true
            },

along with:

            {
                "field": "file.name",
                "include_unmapped": true
            },

is what leads to the issue and matches the description of the elasticsearch bug. There are 2 possible workarounds:

(easy) use timeline, as this code path uses the deprecated but not susceptible to this bug _source part of the response for threat indicator alerts, and so displays as normal

(advanced) edit local storage in your browser:

1. Open chrome/browser of choice dev tools
2. Go to "Application" tab and "Storage" section, the name will be the uri of your kibana instance (circled in red, localhost for me)
3. Copy the value of the key "detection-engine-alert-table-securitySolution-alerts-page-gridView" (in green)
4. This is a json blob used to persist the state of the table, including the selected columns. Edit both the columns and visibleColumns arrays so that the entry with id: 'file.name' is removed from columns and the string 'file.name' is removed from visibleColumns, after doing so mine looks like:

{
  "columns": [
    {
      "initialWidth": 200,
      "columnHeaderType": "not-filtered",
      "id": "@timestamp",
      "schema": "datetime"
    },
    {
      "id": "event.sequence",
      "schema": "numeric"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "displayAsText": "Rule",
      "id": "kibana.alert.rule.name",
      "linkField": "kibana.alert.rule.uuid",
      "schema": "string"
    },
    {
      "initialWidth": 105,
      "columnHeaderType": "not-filtered",
      "displayAsText": "Severity",
      "id": "kibana.alert.severity",
      "schema": "string"
    },
    {
      "initialWidth": 100,
      "columnHeaderType": "not-filtered",
      "displayAsText": "Risk Score",
      "id": "kibana.alert.risk_score",
      "schema": "numeric"
    },
    {
      "initialWidth": 450,
      "columnHeaderType": "not-filtered",
      "displayAsText": "Reason",
      "id": "kibana.alert.reason",
      "schema": "string"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "id": "host.name",
      "schema": "string"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "id": "user.name",
      "schema": "string"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "id": "process.name",
      "schema": "string"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "id": "source.ip",
      "schema": "ip"
    },
    {
      "initialWidth": 180,
      "columnHeaderType": "not-filtered",
      "id": "destination.ip",
      "schema": "ip"
    }
  ],
  "sort": [
    {
      "@timestamp": {
        "order": "desc"
      }
    }
  ],
  "visibleColumns": [
    "@timestamp",
    "event.sequence",
    "kibana.alert.rule.name",
    "kibana.alert.severity",
    "kibana.alert.risk_score",
    "kibana.alert.reason",
    "host.name",
    "user.name",
    "process.name",
    "source.ip",
    "destination.ip"
  ]
}

5. Paste this JSON back into the value column for that key, hit enter, and refresh the page. Alerts table should now work as expected. Just be careful to not re-add 'file.name' to the selected columns via the field browser, as that will get the app back into a bad state:

Third workaround: filter the alerts table to exclude any threat indicator match alerts, unselect the 'file.name' column using the fields browser, and then remove filters. Table should work as expected without 'file.name'. Only an option if you have non threat indicator alerts available, table and field browser will not be visible at all if all alerts in the table are threat indicator on initial load.

Both of these issues will be fixed in upcoming releases, and sorry for the delayed communication on this, these were both pretty brutal to reproduce. I was trying to reproduce #2 just by curling malicious domains instead of downloading actual malicious files ha, took a while to figure out 'file.name' was the culprit.

[nicpenning]

Great sleuthing!! That is awesome! I frequently am changing the text size in my browsers, so I can definitely see that as being the issue. Thanks for diving into this and finding the root cause.

[MakoWish]

@kqualters-elastic,

Adjust the os zoom level settings and refresh the page, or zoom the browser in and out.

Unfortunately, that does not appear to be the case for me on that one. I have never adjusted the zoom level in my browser (Edge Chromium), nor the scaling in the OS. I believe I had mentioned that in my original ticket for that table size issue.

As for the recent one with the table being the correct size but not loading any data, I will have to give that a try.

[kqualters-elastic]

@kqualters-elastic,

Adjust the os zoom level settings and refresh the page, or zoom the browser in and out.

Unfortunately, that does not appear to be the case for me on that one. I have never adjusted the zoom level in my browser (Edge Chromium), nor the scaling in the OS. I believe I had mentioned that in my original ticket for that table size issue.

As for the recent one with the table being the correct size but not loading any data, I will have to give that a try.

There is a more tedious workaround, by using the field browser to remove fields until the width of all columns is less than the page, that should cause the recalculation to happen and hopefully get the alert to display. Can also try adjusting zoom or os level font sizing, as I think the bug will happen at some values, and not at others. There seem to be a whole lot of different ways to trigger this bug with the pixel math, so maybe these workarounds won’t ever work for whatever reason on your particular setup, but the fix in eui should make the height calculation correct in all cases once it’s out.

[kqualters-elastic]

https://github.com/elastic/elasticsearch/pull/97987 has been merged in elasticsearch 8.9.x and main, and i confirmed this fixes the issue:

Will be in 8.9.1+

[MadameSheema]

@sukhwindersingh-qasource @karanbirsingh-qasource can you please validate the fix of the following issue on latest main and 8.9 branches? Please make sure first you are able to reproduce the issue on the above mentioned version. Thanks :)

[ghost]

Hi @MadameSheema

we have tried to first reproduce the issue on mentioned version that is 8.8.0 and on our side issue is not occuring . single alert is readable on the Alert Table and Rule Details Page.

Build Details:

Version: 8.8.0
Commit:2973fcc10d985e4ab94e5eeef976aad0046c6cce
Build:63142

Hardware details: Hardware on which kibana is browsed

Screen-Resolution: 1920x1080
OS: windows
Version: window 10 version 22H2 ( OS Build 19045.3208)

Observation:

• Firefox 115.0.2

• Chrome Version 115.0.5790.110 (Official Build) (64-bit)

[MadameSheema]

Hey team! Any idea about which are the exact steps to repro the issue?

[logeekal]

@MadameSheema @karanbirsingh-qasource , For us, this issue only happens in threat indicator rules with a nested field for example, file.name.

This bug report gives steps to reproduce : https://github.com/elastic/elasticsearch/issues/97684

[ghost]

@logeekal we have followed the steps of https://github.com/elastic/elasticsearch/issues/97684 are we are getting expected error as mentioned in the ticket description on 8.8.0.

However the main issue was of single alert hidden on alert table due to table height is still not reproducible with threat indicator, we are following this test-case for threat indicator https://elastic.testrail.io/index.php?/cases/view/147617

[logeekal]

@karanbirsingh-qasource , Okay I understand. Sorry for the confusion but I do not think that the other issue has been fixed. Maybe @kqualters-elastic can correct me if I am wrong.

[michaelolo24]

@logeekal - yea, that issue is unrelated to this one. @MadameSheema this one is difficult to reproduce and only happened sporadically for us. Some users said it happened more often for them in safari than other browsers, so maybe that's worth a try? If we're unable to reliably reproduce it, can we close this and re-open if the issue shows up again?

[MadameSheema]

@MadameSheema this one is difficult to reproduce and only happened sporadically for us. Some users said it happened more often for them in safari than other browsers, so maybe that's worth a try? If we're unable to reliably reproduce it, can we close this and re-open if the issue shows up again?

Sounds good to me. @karanbirsingh-qasource can you please try to repro on Safari? Thanks!

[MakoWish]

This still seems to be an issue even on 8.9.2.

[kqualters-elastic]

The fix is in eui v85.0.0+ (https://github.com/elastic/eui/blob/v85.0.0/src/components/datagrid/utils/grid_height_width.ts vs https://github.com/elastic/eui/blob/v82.1.0/src/components/datagrid/utils/grid_height_width.ts), and kibana 8.9.x is on eui version v82.1.0 right now for some reason. Not sure how eui versioning works exactly. @cee-chen Do you know if kibana 8.9.x can be upgraded further than v82.1.0, or is that being held back on purpose for some reason?

[cee-chen]

Do you know if kibana 8.9.x can be upgraded further than v82.1.0, or is that being held back on purpose for some reason?

Each individual EUI upgrade PR is usually a large amount of effort in terms of accounting for breaking changes, snapshot changes, test changes/failures, CODEOWNER reviews and so forth. Upgrading EUI 3+ majors on an older Kibana version would take a lot of effort and we'd be extremely reluctant to do so. If we absolutely had to, we would would instead backport the specific requested fix to a new v82.1.1 patch release.

Is there any way we can ask the customer to upgrade Kibana versions instead?

[MakoWish]

Is there any way we can ask the customer to upgrade Kibana versions instead?

We are already on 8.9.2, and a little hesitant to upgrade to 8.10.0, as we typically wait for a .1 patch release (8.10.1 in this case) first.

[MadameSheema]

@karanbirsingh-qasource can you please validate the fix of this issue on the following BC?

[ghost]

Hi @MadameSheema

we have validated this issue on 8.14 BC5 and found the issue not reproducible at our end. Single alert is correctly showing under alert table.

Build Details:

Version: 8.14.0 BC5
Build: 73931
Commit: 7ea00b6178d67183a4def9bdd060b062cced043e

Screen-Shot: