[Inventory] data grid

[smith]

Summary

The new inventory page we should show the list of entities from EEM using the latest index

1. Entity name
2. Entity type
3. Last seen

Designs

[!CAUTION]
Please review the designs on Figma - DESIGNS

(Screenshot taken on 12.09.2024)

Column names and popovers

Column Name	Popover copy
Entity name	Name of the entity (entity. displayName)
Type	Type of entity (entity.type)
Last seen	Timestamp of last received data for entity (entity.lastSeenTimestamp)

Acceptance criteria

• [x] Use EUI data grid (https://eui.elastic.co/#/tabular-content/data-grid)
• [ ] Display 20 entities per page
• [x] Display 500 entities (the user will only be able to paginate through the first 500)
• [x] Above the data grid it shows the total number of the entities
• Entities with multiple datastream types should be listed in separate rows.
• [ ] Sorted by Last seen (default)
• [x] Ability to sort by any column
• [x] The url should store the sorting (sortField, sortDirection)
• [x] When a field is not present do not display anything
• [x] We'll show one row per entity type (Service, host or container)
• [x] We won't display any information about data stream type on the Inventory page
• [x] I'll filter the entity query to only return Service, Host and Container types
• [x] I'll filter the entity definition ID to filter out entities that are not from the built in definition.
• [x] I'll remove the cloud availability zone from the table for now

Columns formatting

• Entity with multiple cloud.availability_zones - https://github.com/elastic/kibana/issues/192975
• Entity name should be clickable - https://github.com/elastic/kibana/issues/192676

I created a separate ticket for formatting the columns, as I think it will be easier to review and test it separately. However, whoever picks this ticket feel free to work on both ticket if you think it's better.

[crespocarlos]

Should the grid be an embeddable?

[kpatticha]

@crespocarlos you mean lens embeddable?

[crespocarlos]

@kpatticha more like if we should also make this inventory grid an embeddable so that people could add it to their dashboards - for the future, obviously

[kpatticha]

@crespocarlos that's a good point but given the time constraints I would suggest to focus on the table, since it's needed anyways and revisit the idea of embeddable later.

Unless you believe we need to do the other way around to avoid technical depth?

[crespocarlos]

@kpatticha Yeah, let's leave it for later.

[smith]

The new inventory page we should show the list of entities from EEM using the latest index

We should specify how we're doing this. I'm assuming we're using an ES client configured with the entity indexes. If possible, we should do search, pagination, and sorting through ES|QL and the filtering through the KQL bar. I guess that means we need a server internal API endpoint?

I asked in #esql about this.

[crespocarlos]

If possible, we should do search, pagination, and sorting through ES|QL and the filtering through the KQL bar.

ES|QL doesn't support pagination yet (see https://github.com/elastic/elasticsearch/issues/100000).

Ability to sort by any column

Are we sorting on the server or the 500 returned items on the client?

[kpatticha]

Are we sorting on the server or the 500 returned items on the client?

• fetch top 500 entities sorted by last seen.
• Client side pagination handled by the EuiDataGrid
• changing the sort instructions or filter triggers a new query.

[kpatticha]

We should specify how we're doing this. I'm assuming we're using an ES client configured with the entity indexes.

Reuse the entity client we created for the new service inventory page.

 const entitiesESClient = await createEntitiesESClient({
      request,
      esClient: coreContext.elasticsearch.client.asCurrentUser,
 });

[kpatticha]

If possible, we should do search, pagination, and sorting through ES|QL and the filtering through the KQL bar. I guess that means we need a server internal API endpoint?

If I'm not mistaken, Caue previously attempted to use ES|QL with the new service inventory page and logs but encountered some limitations. Given the time constraints, it may not be feasible to explore this further right now. I think it would be best to stick with the solutions we already have in place and explore ES|QL in the future, what do you think @smith ?

[dgieselaar]

fwiw, there's references here: https://github.com/elastic/kibana/pull/193082. using ES|QL is fairly easy and you can combine it with ES DSL (and thus KQL).

[miltonhultgren]

Display 500 entities (the user will only be able to paginate through the first 500)

What is this limit based on?

Entities with multiple datastream types should be listed in separate rows.

I don't understand the thinking on this, the goal of using entities is to make the signal types more opaque, it's the same entity so why split it across two rows?

The design shows only host/container/service but this should be built for any type of entity being present in the system, not just based on the built in definitions the stack will ship with.

It looks like the names are links, do they open some kind of details flyout or are they meant to link to another page? If a user has a custom entity definition, what happens to this link? If the user has their own definition for type host, does the entity details page for type host still work? Similarly, I can see that the type row has some custom icons and labelling but that row will have to account for the fact that a user might install a custom definition of an unexpected type. The type filter selector also needs to show all those options.

Also, consider that there may be users who also run a security use case in the same cluster (since EEM is not space aware), you'll end up with whatever entities the Security solution finds as well, so the solution needs to account for this as well.

[dgieselaar]

Why is cloud.availability_zone there and why are we assuming it's a single value for all entity types (it obviously isn't for services and not sure if we can guarantee it for hosts and containers)? And what is the value to the user? It feels like we are just trying to fill up the table with content.

[cauemarcondes]

I don't understand the thinking on this, the goal of using entities is to make the signal types more opaque, it's the same entity so why split it across two rows?

To link to the appropriate page.

The design shows only host/container/service but this should be built for any type of entity being present in the system, not just based on the built in definitions the stack will ship with.

We won't be adding any filter to only display these entities, we just mentioned that because this will be supported at this point.

It looks like the names are links, do they open some kind of details flyout or are they meant to link to another page? If a user has a custom entity definition, what happens to this link? If the user has their own definition for type host, does the entity details page for type host still work?

They will link to each solution, apm/hosts/discover... We're not worried about the edge case at this point.

Similarly, I can see that the type row has some custom icons and labelling but that row will have to account for the fact that a user might install a custom definition of an unexpected type. The type filter selector also needs to show all those options.

The icon is there but we're still evaluating and thinking about this. For this first iteration, I won't be adding any icons.

[cauemarcondes]

Why is cloud.availability_zone there

@roshan-elastic 👆🏻

Why is cloud.availability_zone there and why are we assuming it's a single value for all entity types (it obviously isn't for services and not sure if we can guarantee it for hosts and containers)?

I'm not assuming this will be a single value, I'll ungroup it to show one per line.

[dgieselaar]

I'm not assuming this will be a single value, I'll ungroup it to show one per line.

We'll deal with it in the same way as with e.g. environments in APM? show tags for each value, in the same row?

[dgieselaar]

They will link to each solution, apm/hosts/discover... We're not worried about the edge case at this point.

It's not the edge case, trust me 😄

[cauemarcondes]

We'll deal with it in the same way as with e.g. environments in APM? show tags for each value, in the same row?

No, we'll have a new row for each item.

e.g: foo | central-foo-1 foo | central-foo-2 foo | central-foo-3

[roshan-elastic]

Why is cloud.availability_zone there

@cauemarcondes - Is there a complication with including it? If so, we can remove it.

[cauemarcondes]

The question is not about being easy or not, is more about does it make sense to have it on the table or are we just adding to fill up the table with content?

[roshan-elastic]

It will be helpful for some users - not others. It depends :)

If it's trivial to add it - I'd like to add it. If we can save a bunch of time then I'd prefer to descope it.

[cauemarcondes]

I suggest leaving it out for now.

[roshan-elastic]

Cool let's remove it then @cauemarcondes

[cauemarcondes]

We had a discussion and we agreed:

1. We'll show one row per entity type (Service, host or container)
2. We won't display any information about data stream type on the Inventory page
3. I'll filter the entity query to only return Service, Host and Container types
4. I'll filter the entity definition ID to filter out entities that are not from the built in definition.
5. I'll remove the cloud availability zone from the table for now

[kkurstak]

Hey, thank you @kpatticha for the update on Figma.

[cauemarcondes]

closed by https://github.com/elastic/kibana/pull/193272