dart-lang / sdk

The Dart SDK, including the VM, JS and Wasm compilers, analysis, core libraries, and more.
https://dart.dev
BSD 3-Clause "New" or "Revised" License
10.09k stars 1.56k forks source link

add an API to export Dart VM metrics such as process cpu seconds total, process threads,process open fds, heap bytes #53831

Open insinfo opened 11 months ago

insinfo commented 11 months ago

add an API to export Dart VM metrics such as process_cpu_seconds_total, process_threads,process_open_fds, process_heap_bytes. both C#.NET and Java expose this information, this would facilitate integration with monitoring tools such as Prometheus

I don't know if this issue is related https://github.com/dart-lang/sdk/issues/49062

https://github.com/dotnet/runtime/blob/c345959e132409b19c692807e313f739bce850ad/src/libraries/System.Diagnostics.Process/src/System/Diagnostics/Process.Linux.cs#L130 https://programtalk.com/vs4/csharp/hanabi1224/Programming-Language-Benchmarks/bench/tool/Linux/procfs/Interop.ProcFsStat.TryReadStatusFile.cs/

https://prometheus.io/docs/instrumenting/writing_clientlibs/#standard-and-runtime-collectors

mkustermann commented 10 months ago

Could this be made as a package using our C FFI to obtain information from various operating systems?

insinfo commented 10 months ago

I think this information has to be exposed by the dart sdk, things like information about the current process, about the garbage collector, CollectionCount, MaxGeneration etc., it would be extremely difficult to reinvent the wheel for this since the dart Vm is already written in c++ and has direct access to the APIs of the operating systems where it runs

Dart today has a ProcessInfo class in dart:io with the current resident set size of memory and maxRss, it would be a matter of adding other properties to this class totalProcessorTime, handleCount , threadsCount , GC collection count etc

.NET exposes this information in the Process class of the System.Diagnostics namespace https://github.com/dotnet/runtime/blob/fe043b489f69a888ad2efd97fbc177c1f25e541d/src/libraries/System.Diagnostics.Process/ref/System.Diagnostics.Process.cs#L118

mkustermann commented 10 months ago

totalProcessorTime, handleCount , threadsCount it would be extremely difficult to reinvent the wheel for this since the dart Vm is already written in c++ and has direct access to the APIs of the operating systems where it runs

Those can all be obtained by asking the system (e.g. procfs on linux). I think it would be wonderful if the community made a package for exposing these things. If there's anything missing in our infrastructure that prevents that, we'd be very eager to know about it (we strive to provide mechanism to empower users to do this - e.g. by providing a powerful C FFI).

We prefer to have dart:* libraries expose things that either couldn't be otherwise implemented by users in packages or things that (if written by users in packages) couldn't be optimized / made fast by our compilers. There's several reasons for this:

(A side note: The Dart VM is embeddable by other code, that means e.g. not all threads / not all memory / ... in the process are owned by the Dart VM)

GC collection count etc

We actually expose something like this: A logical GC clock that moves forward when: dart:developers reachabilityBarrier. I don't think we want to expose very detailed information from the VM internals as explicit APIs - as those implementation details may change over time, rendering APIs obsolete. We do expose somewhat more informatino via our vm-service websocket protocol though.