search

bloomberg / memray

Memray is a memory profiler for Python
https://bloomberg.github.io/memray/
Apache License 2.0
13.17k stars 392 forks source link

Track virtual memory #571

Closed acampove closed 6 months ago

acampove commented 6 months ago

Is there an existing proposal for this?

Is your feature request related to a problem?

I am sending jobs to computing clusters, which limit the memory usage but they do not count the physical memory, instead the clusters count the virtual memory. However memray seems to only track physical memory usage.

Describe the solution you'd like

I need a way to find out what and where my process is using virtual memory.

Alternatives you considered

No response

pablogsal commented 6 months ago

Hi @acampove and thanks for opening an issue. Unfortunately there isn't a portable way to assign virtual memory consumption in a performant way to every allocation request so we won't be able to do offer this in memray. We are already tracking resident memory, we can try to also include other metrics in the thread but this won't allow you to know where is being requested (although it will correlate with allocations).