[Request]: Update Containers docs for Infrastructure

roshan-elastic commented 5 months ago

Description

We're going to be updating our container template to (a) use the new template we are using for hosts (b) making 'docker containers' support all containers.

Initially, this means we need to:

Update the metrics definitions
Update the inventory docs to reflect these are 'containers' (not just 'docker containers')

Old view of containers containers - now

What it will start to look like (showing 'hosts' for now) containers - next

Notes:

See Docker Container Metric Definitions and Kubernetes Container Metric Definitions
There will be future releases which start to incorporate 'containers' into other views (e.g. from a host or a service) but that's out of scope for now (see epic for details)

Resources

[x] https://github.com/elastic/observability-dev/issues/3313

Which documentation set does this change impact?

Stateful and Serverless

Feature differences

None

What release is this request related to?

TBC (likely 8.15)

Collaboration model

I'm happy to help!

Point of contact.

Main contact: @roshan-elastic + @jennypavlova + @MiriamAparicio

Stakeholders: @smith

roshan-elastic commented 5 months ago

@colleenmcginnis FYI

bmorelli25 commented 5 months ago

Thanks for opening this issue. Marking as blocked and adding to our backlog until we know those Docker metric definitions.

roshan-elastic commented 3 months ago

Hey @bmorelli25,

We've completed this issue now (barring a minor bug with the logs tab) but hopefully this is good to start documenting?

I'd imagine you'll need some support untangling this.

Here is a summary:

Containers in Inventory
Container Views

Containers in Inventory

There are no changes to the UI behaviour for 'Docker Containers' in Inventory except for clicking on one will open the new 'Container View' fly-out (see 2. Container Views below).

Details

In Inventory, all container types will continue to show in the waffle map (they show all container runtimes, e.g. Docker, containerd - anything which emits container.id):

However, there is a 'bug' which we can't get around for the moment which is that the Inventory UI says Docker Containers when we are in fact showing ALL containers (this has been the case for a long time).
This works fine if you happen to only have 'docker' containers, the UI works OK - including the metrics you see per container:

Note : that the 'metrics' used for the 'Docker Containers' are using docker specific fields which only work with docker containers:

However, if you are looking at non-docker containers, the metrics will all show as '0%' or '-' because the docker fields don't exist:

We do plan to move towards a single 'Container' entity type which works for any runtime (i.e. Docker and other runtimes) but we can't get there yet

Containers Views

We have updated the container view to be similar to 'Hosts', i.e. there is a fly-out and an updated container view:

Details

Metrics

Even when there are no docker metrics, the UI will adapt to show the correct metrics based on the runtime (this was the case in the old view too).

At the moment, it will adapt to either docker metrics or Kubernetes metrics (e.g. average(kubernetes.container.cpu.usage.limit.pct)).

Note : You can test by opening any visualisation in Lens but I'm sure @jennypavlova can help

Alerts This is a new section:

Any alert which matches the container.id should show in the list
If you are viewing a 'Docker' container then you can create an alert against it:

If you are viewing any other container runtime, you will not be able to create an alert against it:

Note : This is because the 'Inventory' alert rule for 'Docker Containers' will only evaluate 'Docker' metrics (so it doesn't make sense to show metrics which won't work for the current container).

Why aren't we just showing 'Containers' instead of 'Docker Containers'?

We want to get there but we need to figure out a way to ensure there is a single 'Container' entity that we can show in the UI and create alerts on.

We could change this today if we wanted but the problem is that the inventory 'Docker Container' entity type with the respective 'metrics' is shared between 'Inventory' and the 'Inventory Alerts':

Inventory - 'Docker Containers' and their metrics

Inventory Alerts - 'Docker Containers' and their metrics

However, if we were to 'docker containers' to 'containers' and make changes to the default metrics, this would impact existing inventory rules users may have. So, instead, we're moving cautiously and we're figuring out how to have a single 'container' entity type with normalised metrics which work with any container type.

roshan-elastic commented 3 months ago

cc @jennypavlova are you able to assist the docs team at all on this?

I'm thinking mainly about field names/usage for the container views as it adapts to docker vs K8s containers.

roshan-elastic commented 3 months ago

(if they reach out that is)

jennypavlova commented 3 months ago

cc @jennypavlova are you able to assist the docs team at all on this?

Sure, please let me know if you have any questions, I am happy to help :)

roshan-elastic commented 2 months ago

Hey @bmorelli25,

Just wanted to check if this one will have some assigned?

It's not urgent but basically making container views.

bmorelli25 commented 2 months ago

Hey @roshan-elastic, I have this on our list, but it'll need to take a backseat to ECO and OTel right now. I don't have an estimated date or writer yet.

roshan-elastic commented 2 months ago

No problem @bmorelli25 - not a priority.

bmorelli25 commented 1 month ago

@roshan-elastic do we know what the metrics definitions are?

Update the metrics definitions...we don't know what these are yet

dedemorton commented 1 month ago

@roshan-elastic To add to Brandon's question, I have a request. Can you quickly update this issue to reflect the current state of this request. For example, I'm assuming that this statement is no longer true:

This is WIP so we'll let you know when we have our first customer-facing release

I also want to make sure we're targeting the correct release. Currently the issue indicates that we are likely targeting 8.15:

TBC (likely 8.15)

roshan-elastic commented 1 month ago

Hey @bmorelli25 @dedemorton,

Can you quickly update this issue to reflect the current state of this request

Sure.

I also want to make sure we're targeting the correct release. Currently the issue indicates that we are likely targeting 8.15:

That's right. This change went out in 8.15.

@roshan-elastic do we know what the metrics definitions are?

Yes, we have them now.

When a container is using Docker, it should be using these definitions:

Docker Metric Definitions

When a container is using ContainerD (i.e. Kubernetes) then it should be using these metrics:

Kubernetes Container Metric Definitions

Note The view of a service should adapt based on the available metrics for the container

Question

Would it make sense to repurpose the https://www.elastic.co/guide/en/observability/current/docker-container-metrics.html page into a more generic container page which as sections for Docker Metrics and Kubernetes Container Metrics?

We have short links in the UI which I can remotely change which are pointing towards Kubernetes or Docker metrics in a page (so you can change the URL without breaking any links in the UI - I can change the short-links to point to anywhere):

e.g.

(where the #VALUE would refer to the id=VALUE in the doc)

https://ela.st/docs-infra-k8s-metrics#key-metrics-memory https://ela.st/docs-infra-k8s-metrics#key-metrics-cpu

https://ela.st/docs-infra-docker-container-metrics#key-metrics-cpu https://ela.st/docs-infra-docker-container-metrics#key-metrics-memory

What I'm thinking is that if we had a single 'container' page with sections for Docker and K8s that I could point towards then https://ela.st/docs-infra-k8s-metrics#key-metrics-memory could point towards YOURCHOSENURL#ID_OF_SECTION?

More information for the future

TL;DR : We're going to normalise all container types in the future to use the same fields for metrics (e.g. container.cpu.usage instead of specific ones per runtime like kubernetes.container.memory.usage.limit.pct for Kubernetes and docker.cpu.total.pct). It's worth bearing in mind that when we make this change, we'll need another docs update to reflect this.

The UI is pretty inconsistent with containers because we're conditionally changing the metric formulae in the container view based on the available metrics. For example, the fields used to show 'CPU' are different between Docker containers and K8s containers.

At the same time, when you go to Inventory, it shows 'Docker Containers' in the drop-down but it will show all containers with a container.id (inc containerd ones for K8s).

This is described in more detail in this epic where we're going to eventually normalise all containers as "Containers" (instead of having references to "Docker Containers" with specific docker metrics) and use the generic container fields like container.cpu.usage.

We'll make all of these changes so that there are just 'Containers' which use generic fields for metrics and then we'll probably leave the existing metrics (which are used in Inventory and Inventory alert rules) as legacy Docker CPU etc.

dedemorton commented 1 month ago

Would it make sense to repurpose the https://www.elastic.co/guide/en/observability/current/docker-container-metrics.html page into a more generic container page which as sections for Docker Metrics and Kubernetes Container Metrics?

@roshan-elastic Sure we can have a single page that contains sections for both docker and K8s metrics.

elastic / observability-docs