search

facebookresearch / dinov2

PyTorch code and models for the DINOv2 self-supervised learning method.
Apache License 2.0
8.98k stars 790 forks source link

PyTorch 2.1 support #265

Open schmidt-ai opened 1 year ago

schmidt-ai commented 1 year ago

Looks like there are some breaking changes to the FSDP API in PyTorch 2.1.

For example, dinov2.fsdp.__init__.py::free_if_fsdp is broken when using torch==2.1: AttributeError: 'DinoVisionTransformer' object has no attribute '_handles'

qasfb commented 1 year ago

Thanks for reporting ! We are still working on the transition to PT2.1 @patricklabatut

schmidt-ai commented 1 year ago

Glad to hear it. Lmk if I can help.

dsuess commented 11 months ago

I found the following works in pytorch 2.1:

def free_if_fsdp(x: FSDP):
    if is_sharded_fsdp(x) and x._has_params:
        handle = x._handle
        _reshard(x, handle, True)