search

MoseleyBioinformaticsLab / gpu_tracker

Context manager and CLI that tracks the computational-resource-usage of a code block or shell command, particularly the GPU usage.
Other
4 stars 1 forks source link

Track child process information #11

Closed erikhuck closed 2 months ago

erikhuck commented 3 months ago

It could be helpful to not only track the maximum gpu ram and ram over the main process and its children but to also give more granular data i.e. the ram of the main process alone followed by the individual measurements for each of its children. It could have a dictionary that maps the "main" process and each of its children (by process ID) to a dictionary containing a "max_ram" and "max_gpu" field. The overall max_ram and max_gpu can then be calculated from that dictionary rather than being calculated directly.

The tracker should probably have a verbose flag that displays just the summed information vs. the more granular version.

hunter-moseley commented 3 months ago

Good idea!

On Mon, Mar 25, 2024 at 11:32 AM Erik Huckvale @.***> wrote:

It could be helpful to not only track the maximum gpu ram and ram over the main process and its children but to also give more granular data i.e. the ram of the main process alone followed by the individual measurements for each of its children. It could have a dictionary that maps the "main" process and each of its children (by process ID) to a dictionary containing a "max_ram" and "max_gpu" field. The overall max_ram and max_gpu can then be calculated from that dictionary rather than being calculated directly.

— Reply to this email directly, view it on GitHub https://github.com/MoseleyBioinformaticsLab/gpu_tracker/issues/11, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEP7BZZQK4IIFK6O4MOMUTY2A7SJAVCNFSM6AAAAABFHHPAFKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGIYDMMBUGY4TOOA . You are receiving this because you are subscribed to this thread.Message ID: @.***>

-- Hunter Moseley, Ph.D. -- Univ. of Kentucky Professor, Dept. of Molec. & Cell. Biochemistry / Markey Cancer Center / Institute for Biomedical Informatics / UK Superfund Research Center Not just a scientist, but a fencer as well. My foil is sharp, but my mind sharper still.

Email: @. (work) @. (personal) Phone: 859-218-2964 (office) 859-218-2965 (lab) 859-257-7715 (fax) Web: http://bioinformatics.cesb.uky.edu/ Address: CC434 Roach Building, 800 Rose Street, Lexington, KY 40536-0093

erikhuck commented 3 months ago

This actually may not be the most viable in the case that a user's main process spawns several different child processes, constantly closing one and starting another. There could be information collected for very many different child processes which only existed for a relatively brief period of time. If we do implement this, we may want to include a warning in the documentation informing them that a lot of memory could be used by the Tracker itself if their process spawns a bunch of child processes so they can choose to keep this setting as False if desired.

hunter-moseley commented 3 months ago

Agree.

On Fri, Mar 29, 2024 at 10:49 AM Erik Huckvale @.***> wrote:

This actually may not be the most viable in the case that a user's main process spawns several different child processes, constantly closing one and starting another. There could be information collected for very many different child processes which only existed for a relatively brief period of time. If we do implement this, we may want to include a warning in the documentation informing them that a lot of memory could be used by the Tracker itself if their process spawns a bunch of child processes so they can choose to keep this setting as False if desired.

— Reply to this email directly, view it on GitHub https://github.com/MoseleyBioinformaticsLab/gpu_tracker/issues/11#issuecomment-2027342984, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADEP7B3LGKAZRXGPU3SM7UDY2V5PPAVCNFSM6AAAAABFHHPAFKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRXGM2DEOJYGQ . You are receiving this because you commented.Message ID: @.***>

-- Hunter Moseley, Ph.D. -- Univ. of Kentucky Professor, Dept. of Molec. & Cell. Biochemistry / Markey Cancer Center / Institute for Biomedical Informatics / UK Superfund Research Center Not just a scientist, but a fencer as well. My foil is sharp, but my mind sharper still.

Email: @. (work) @. (personal) Phone: 859-218-2964 (office) 859-218-2965 (lab) 859-257-7715 (fax) Web: http://bioinformatics.cesb.uky.edu/ Address: CC434 Roach Building, 800 Rose Street, Lexington, KY 40536-0093