lmolkova
commented
6 months ago tagging the owners - @open-telemetry/semconv-system-approvers, could you please take a look?
Open TimothyMothra opened 6 months ago
lmolkova
commented
6 months ago tagging the owners - @open-telemetry/semconv-system-approvers, could you please take a look?
TimothyMothra
commented
6 months ago I'm continuing to investigate what metrics are available in other languages and it seems that "physical bytes" is consistently available.
.NET
Python
Node
I'm looking for some clarity on the Process Metrics. Specifically, why is "process.memory.usage" defined as "physical memory"? https://github.com/open-telemetry/semantic-conventions/blob/main/docs/system/process-metrics.md#metric-processmemoryusage
It seems to me that "memory.usage" is vague, and it's not clear to me why this was assigned "physical memory".
I found the original PR and I can see from the conversation that this was originally:
And then and some point the PR was changed to:
Proposal
As this spec is still Experimental, is there appetite to make changes? I propose for consideration:
Example for Discussion
I'm a .NET developer and I'm specifically interested in the Private Bytes. This is the value that various other tools show me, but it also seems to be more representative of the total memory currently allocated.
Sample code to generate memory pressure (CLICK TO EXPAND)
```csharp long totalMemoryAllocated = 0; int chunkSizeMB = 1000; try { while (true) { // Allocate memory in chunks byte[]? memoryChunk = new byte[chunkSizeMB * 1024 * 1024]; // 1 MB = 1024 * 1024 bytes totalMemoryAllocated += memoryChunk.Length; // Release the allocated memory memoryChunk = null; System.Threading.Thread.Sleep(500); } } catch (OutOfMemoryException) { Console.WriteLine("Out of memory!"); } ```Visual Studio's debugger Diagnostic Tools shows me Private Bytes.
Process Explorer shows me Private Bytes
Terminology
While investigating this, it seems that there are conflicting definitions depending on where you read. If someone has some preferred definitions of the different types of memory, or examples from other languages please share in the comments! :)
.NET provides some metrics with their own definitions:
Asking ChatGPT to define (CLICK TO EXPAND)
**You** Can you provide a comparison/contrast of the following types of memory: public, private, committed, and virtual **ChatGPT** Certainly! Let's compare and contrast the following types of memory: public, private, committed, and virtual memory. 1. Public Memory: - Public memory refers to memory that is accessible and shared among multiple processes or users. - It is commonly used in multi-user systems or environments where processes need to share data. - Public memory facilitates communication and data exchange between different processes or users. - Example: Shared libraries in an operating system that are accessible to multiple processes. 2. Private Memory: - Private memory refers to memory that is allocated exclusively to a single process and cannot be accessed by other processes. - Each process has its own private memory space, ensuring data isolation and security. - Private memory protects the integrity of data within a process and prevents unintended access or modification by other processes. - Example: Stack and heap memory allocated to individual processes in an operating system. 3. Committed Memory: - Committed memory refers to the portion of virtual memory that has been allocated to a process and is backed by physical storage, such as RAM or disk space. - When a process requests memory, the operating system commits a portion of virtual memory to fulfill that request. - Committed memory may not necessarily be actively used; it simply indicates that the memory has been reserved for a specific process. - Example: When a program loads into memory, the operating system commits a certain amount of virtual memory to accommodate its requirements. 4. Virtual Memory: - Virtual memory is a memory management technique that provides an illusion of infinite memory by allowing the operating system to use both RAM and disk storage to simulate additional memory. - It allows processes to access more memory than physically available by swapping data between RAM and disk as needed. - Virtual memory enables efficient memory utilization and multitasking by allowing multiple processes to run simultaneously without exhausting physical memory resources. - Example: When RAM becomes scarce, the operating system may move less frequently accessed data from RAM to disk storage, freeing up physical memory for more active processes. Comparison: - Public and private memory differ primarily in terms of accessibility and sharing among processes. Public memory is shared, while private memory is exclusive to individual processes. - Committed memory is a subset of virtual memory, representing the portion that has been allocated to processes and backed by physical storage. - Virtual memory encompasses the entire memory management system, including both physical RAM and disk storage, providing a mechanism for extending available memory beyond physical limits. Contrast: - Public and private memory differ in their access control mechanisms: public memory allows access to multiple processes, while private memory is restricted to individual processes. - Committed memory specifically refers to memory that has been allocated and backed by physical storage, while virtual memory is a broader concept encompassing the entire memory management system, including both physical and virtual storage. - Public and private memory deal with the organization of memory within processes, while committed and virtual memory focus on the management and allocation of memory resources by the operating system.