Closed dujiangsu closed 2 years ago
By the way, I notice some interesting stuff. When executing inference with large batch size, the memory usage can extend to 56GB (30B model). However, if I fix 48GB in advance, as in the picture, the practical memory usage is not that large.
By the way, I notice some interesting stuff.
When executing inference with large batch size, the memory usage can extend to 56GB (30B model). However, if I fix 48GB in advance, as in the picture, the practical memory usage is not that large.