Closed HarrytheOrange closed 2 years ago
I run inference on 1060 6G, after several usages, the programme can take about 5Gb GPU memory after running but before running the inference. torch.cuda.empty_cache did not work.
Set reduce seq_chunk or set seq_chunk=1 will help.
thanks so much
I run inference on 1060 6G, after several usages, the programme can take about 5Gb GPU memory after running but before running the inference. torch.cuda.empty_cache did not work.