search

GSTT-CSC / project-template

Template for new projects
0 stars 0 forks source link

Explore using nvidia NGC pytorch container and AMP #3

Closed laurencejackson closed 2 years ago

laurencejackson commented 3 years ago

nvidia produce a containerised version of pytorch on NGC which has native automatic mixed precision (AMP) support by default. AMP mixes full precision float32 values with half-precision float16 values to speed up training and reduce the memory cost of complex models. This could be an interesting way to speed up models.

https://ngc.nvidia.com/catalog/containers/nvidia:pytorch

laurencejackson commented 3 years ago

https://pytorch.org/blog/accelerating-training-on-nvidia-gpus-with-pytorch-automatic-mixed-precision/

laurencejackson commented 2 years ago

Using nvidia ubuntu dev images as of #40. AMP available through precision kwarg in pytorch lightning trainer class