🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support
Since downcasting on TPUs is better for calculating metrics and logging values, it might be nice to have an API that lets you toggle downcasting via a context manager.
Under the hood this should use patch_environment to patch the right environmental variable.
Proposed API design:
with accelerator.tpu_downcast():
# Run through evaluation loop
Since downcasting on TPUs is better for calculating metrics and logging values, it might be nice to have an API that lets you toggle downcasting via a context manager.
Under the hood this should use
patch_environment
to patch the right environmental variable.Proposed API design: