search

pesser / edflow

Reduce boilerplate code for your ML projects. TensorFlow and PyTorch.
https://edflow.readthedocs.io/
MIT License
24 stars 13 forks source link

GPU utilization bar #248

Open theRealSuperMario opened 4 years ago

theRealSuperMario commented 4 years ago

Sometimes it would be nice to know if the gpu is under-utilized, indicating that the I/O bandwidth limits the GPU.

I am not sure if this is easy to measure, because one might need to figure out if the gpu is idle at some point.

jhaux commented 4 years ago

I agree that this would be super cool to have in the iterator interface, right above the epoch bar for example. But there already is an easy way to do this: Wandb allows you to track system parameters. Sadly it logs all gpus, so you have to take care to only look at the one you are interested in. E.g. if you use CUDA_VISIBLE_DEVICES=4 you need to take a look at the logs under system/gpu.4.<parameter>. system_log