search

pytorch / examples

A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
https://pytorch.org/examples
BSD 3-Clause "New" or "Revised" License
22.45k stars 9.55k forks source link

Fix AC in T5 example #1273

Closed msaroufim closed 5 months ago

msaroufim commented 5 months ago

This fixes https://github.com/pytorch/examples/issues/1210

Enabling AC is enabling compile by default which is then failing to handle the cpu offloading flag for FSDP?

This is broken on 2.2, 2.3 and nightlies

cc @awgu @anijain2305

netlify[bot] commented 5 months ago

Deploy Preview for pytorch-examples-preview canceled.

Name Link
Latest commit d16a5c23f13fcd9372a83d843aa6f61647c33c0e
Latest deploy log https://app.netlify.com/sites/pytorch-examples-preview/deploys/667fa9009bec590008d0a754
awgu commented 4 months ago

I think the arg signature changed for the checkpoint wrapper. We should remove this line: https://github.com/pytorch/examples/blob/26de41904319c7094afc53a3ee809de47112d387/distributed/FSDP/policies/activation_checkpointing_functions.py#L16