Using --no-cuda option still uses GPU memory

[tungnt55]

Bug description Running BlenderBot2 in interactive mode with --no-cuda option. GPU utilization is at 0~1% but the program still uses GPU memory.

Reproduction steps Running BlenderBot 2 with the following command: parlai interactive --model-file zoo:blenderbot2/blenderbot2_400M/model --search-server None --knowledge-access-method memory_only --memory-key full_text --inference topk --no-cuda

Expected behavior The program should not use any GPU memory.

Logs Please paste the command line output: Below is the nvidia-smi output

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.103.01   Driver Version: 470.103.01   CUDA Version: 11.4     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:02:00.0  On |                  N/A |
|  0%   59C    P2    33W / 170W |   8083MiB / 12031MiB |      1%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1118      G   /usr/lib/xorg/Xorg                 53MiB |
|    0   N/A  N/A      1674      G   /usr/lib/xorg/Xorg                294MiB |
|    0   N/A  N/A      1819      G   /usr/bin/gnome-shell               65MiB |
|    0   N/A  N/A      3188      G   /usr/lib/firefox/firefox           10MiB |
|    0   N/A  N/A    170635      C   python                           7627MiB |
+-----------------------------------------------------------------------------+

Output of running parlai:

11:31:55 | Overriding opt["model_file"] to /home/tung/Desktop/ros/haru_chat_ws/src/tools/data/models/blenderbot2/blenderbot2_400M/model (previously: /checkpoint/kshuster/projects/knowledge_bot/kbot_memfix_sweep25_Fri_Jul__9/338/model.oss)
11:31:55 | Overriding opt["knowledge_access_method"] to memory_only (previously: classify)
11:31:55 | Overriding opt["memory_key"] to full_text (previously: personas)
11:31:55 | Overriding opt["inference"] to topk (previously: beam)
11:31:55 | Overriding opt["no_cuda"] to True (previously: False)

Additional context Add any other context about the problem here. (like proxy settings, network setup, overall goals, etc.)

[klshuster]

Thanks for flagging, the fix is in #4377

[tungnt55]

Thank you for the fix! I confirm that it is working properly now.