lmassaron
commented
3 years ago Probably the way data is feeded into the network has to be revamped by using tf.data. Also it depends on the batch size you are using, the larger, the more utilization you will have because that will require larger matrix multiplications. I've wrote about using tf.data in a tabular pipeline on my most recent book -> https://www.amazon.com/Machine-Learning-Using-TensorFlow-Cookbook-ebook/dp/B08V8WC9W8/ have a look at it if you can get a copy of the book because I don't know when I will update the code of this repository.
Fri Jan 29 23:29:47 2021
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 455.38 Driver Version: 455.38 CUDA Version: 11.1 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |===============================+======================+======================| | 0 GeForce RTX 3090 Off | 00000000:01:00.0 Off | N/A | | 32% 32C P8 23W / 350W | 23081MiB / 24245MiB | 0% Default | | | | N/A | +-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 27737 C ...rtualenvs/tf24/bin/python 23079MiB | +-----------------------------------------------------------------------------+
I checked that my GPU is available. I think my GPU always waits for CPU to process data. Do you know how to improve the utility of GPU? I tried your example, most of the time, GPU-Util was 0%, sometimes, it showed 20%.