Closed carterbox closed 10 months ago
Prevent GPU memory from being allocated at import time on the 0th GPU.
Streams take GPU memory and may be unused if only non-GPU functions are used, so wait until functions are called to do initialization.
Purpose
Prevent GPU memory from being allocated at import time on the 0th GPU.
Approach
Streams take GPU memory and may be unused if only non-GPU functions are used, so wait until functions are called to do initialization.
Pre-Merge Checklists
Submitter
Reviewer