open-telemetry / semantic-conventions

Defines standards for generating consistent, accessible telemetry across a variety of domains
Apache License 2.0
250 stars 161 forks source link

Add kernel related semantic conventions #66

Open mx-psi opened 1 year ago

mx-psi commented 1 year ago

I would like to add three new resource-level semantic conventions to the os.* namespace that include information about the kernel used by the system. Roughly, they could be defined as follows:

Attribute Type Description Examples Requirement Level
os.kernel.name string The kernel name, e.g. on a Unix system, the output of uname -s. Linux Recommended
os.kernel.release string The kernel release, e.g. on a Unix system, the output of uname -r 5.19.0-42-generic Recommended
os.kernel.version string The kernel version, e.g. on a Unix system, the output of uname -v #43~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Apr 21 16:51:08 UTC 2 Recommended

These would be useful to identify a resource as coming from a machine with a specific kernel version, which can be useful in debugging and troubleshooting systems.

It's been a while since I have contributed to semantic conventions so I have a few open questions, please bear with me :)

Open questions

  1. Elastic Common Schema already defines os.kernel as "Operating system kernel version as a raw string." and gives 4.4.0-112-generic as an example value (so, effectively what I called here os.kernel.release). I think we should use the more specific os.kernel.release for this instead, since os.kernel seems more like a namespace to me, and there's several meanings for 'version'. Is that okay?
  2. Should os.kernel.name have a pre-defined set of values? If so, what values are in scope?
  3. Are the values as described sufficiently cross-platform? At a minimum, do they have a reasonable equivalent on Windows and macOS?
  4. Is the proposed description well-specified enough? Are uname output and flags consistent enough to use it as a standard? Based on the description of host.id it sounds like it should be okay, but I am happy to consider a different definition if something is suggested.
mx-psi commented 1 year ago

Some notes:

  1. Should os.kernel.name have a pre-defined set of values? If so, what values are in scope?

I think a reasonable start is to specify the values for the kernels from os.type.

  1. Are the values as described sufficiently cross-platform? At a minimum, do they have a reasonable equivalent on Windows and macOS?

uname on macOS and winver on Windows can provide this information. os.kernel.release can be contentious on Windows (e.g. as shown on this Wikipedia table version has stayed as '10.0' even on Windows 11)

mx-psi commented 9 months ago

@ChrsMark @AlexanderWert @trisch-me would be interested to know your thoughts about this one. Unfortunately os.kernel is in ECS, and here I was proposing using it as a namespace. Is os.kernel widely used in ECS? How bad the breakage would be? Do you have suggestions for an alternative namespace name?

trisch-me commented 9 months ago

Hi @mx-psi, I have checked we are using os.kernel alone in security in around 100 different integrations. Saying that I like this proposal to have os.kernel as a namespace. For us it would be a breaking change to use os.kernel.release instead of just os.kernel. I'm checking now on how big would be this breakage for us

trisch-me commented 9 months ago

Hey @mx-psi After some discussions within Elastic, I would say that changes, which involve not just the name but the actual data type (transitioning from a leaf node to an object), are highly undesirable. They could make the correct adoption of new fields from Otel in the future almost unfeasible due to potential mapping conflicts. Changing just a name is also a breaking change but we can work it out.

Taking this into account and also following our soon to be merged guidelines I propose to use new naming for this namespace. We could use something like system_kernel, kernel_info or kernel_data, maybe operating_kernel.

Additional consideration: I'm planning to expand the os namespace and add additional fields from ECS. We can defer this discussion until I create a pull request, but we'll need to decide whether to adopt the ECS os.kernel field.

mx-psi commented 9 months ago

I'm planning to expand the os namespace and add additional fields from ECS. We can defer this discussion until I create a pull request, but we'll need to decide whether to adopt the ECS os.kernel field.

Sure, we can defer up until that point, please ping me in the PR :) I will think about possible alternatives for the namespace from my side as well

trisch-me commented 5 months ago

hey @mx-psi I think we can bring this issue up again. I have checked and there are no other fields for OS except os.family, os.kernel, os.platform and os.full (Operating system name, including the version or code name.)

So I will not create any new PR because these are exact fields we need to make agreement on

mx-psi commented 5 months ago

@trisch-me Okay! To me, the main question here is what namespace to use if os.kernel is not a possibility because of ECS limitations. I don't love any of the options on https://github.com/open-telemetry/semantic-conventions/issues/66#issuecomment-1854415044 but I don't have better ideas either

cc @open-telemetry/semconv-system-approvers

ChrsMark commented 5 months ago

Do we have examples of how other operating systems will populate these attributes? If this detailed info is only for unix based systems we could use sth like os.unix_kernel.*?

I also wonder if the usage of os.kernel in ECS is accurate when it only includes the version but the truth is that when we ask "what kernel are you running on?" we answer with the version usually, right?

Update

On a mac we have:

~ ❯ uname -r                                                                                                                                                                      
23.4.0
~ ❯ uname -s                                                                                                                                                                      
Darwin
~ ❯ uname -v                                                                                                                                                                      
Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:x-10063.101.17~1/RELEASE_ARM64_T6000
~ ❯ uname -a                                                                                                                                                                      
Darwin H471B7HXWP 23.4.0 Darwin Kernel Version 23.4.0: Fri Mar 15 00:10:42 PDT 2024; root:x-10063.101.17~1/RELEASE_ARM64_T6000 arm64
trisch-me commented 5 months ago

From what I understand in ECS os.* namespace is already a top namespace for all underlying OS attributes such as name and kernel and version etc. So maybe we should take it as is and not introduce a specific namespace for kernel?