MTCNN not using GPU

[xaviermerino]

This is a similar thread to that of #157. I figured since that one was old, I would start a new one.

It seems like Fawkes is not using MTCNN with GPU. After going through the directory of images and reporting the number of images in there, it is slowly aligning and detecting faces. See below:

1 Physical GPUs, 1 Logical GPU
Downloading data from http://mirror.cs.uchicago.edu/fawkes/files/extractor_2.h5
161832960/161829576 [==============================] - 31s 0us/step
Identify 24857 files in the directory
Identify 24857 images in the directory
Find 1 face(s) in 201761_00F27.JPG
Find 1 face(s) in 072462_10F43.JPG

And when checking the CPU utilization, is pegged at 100% while my GPU is at 0% even though the process has allocated GPU memory.

|===============================+======================+======================|
|   0  NVIDIA RTX A6000    Off  | 00000000:01:00.0 Off |                  Off |
| 45%   72C    P2    98W / 300W |   1977MiB / 49140MiB |      0%      Default |
|                               |                      |                  N/A |

Any ideas of what can be done @Shawn-Shan @emilywenger ?

Update: Once it is done with MTCNN, Fawkes is using GPU as expected.

|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA RTX A6000    Off  | 00000000:01:00.0 Off |                  Off |
| 60%   83C    P2   173W / 300W |   5561MiB / 49140MiB |     51%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+