search

pytorch / tutorials

PyTorch tutorials.
https://pytorch.org/tutorials/
BSD 3-Clause "New" or "Revised" License
8.23k stars 4.07k forks source link

[BUG] - <title>When I use fsdp, Because the flattened parameters, I always meet some question #2610

Open sqzhang-lazy opened 1 year ago

sqzhang-lazy commented 1 year ago

Add Link

When I use fsdp, Because the flattened parameters, I always meet some question. for examples: RuntimeError: mat2 must be a matrix, got 1-D tensor and RuntimeError: weight should have at least three dimensions It always occurred in some flattened model weights, sucn as conv, linear etc. How can I solve this problem?

Describe the bug

When I use fsdp, Because the flattened parameters, I always meet some question for examples: RuntimeError: mat2 must be a matrix, got 1-D tensor and RuntimeError: weight should have at least three dimensions It always occurred in some flattened model weights, sucn as conv, linear etc. How can I solve this problem?

Describe your environment

Pytorch 2.1.0

cc @osalpekar @H-Huang @kwen2501

yiliu30 commented 1 year ago

Hi @648652443, could please provide some code snippets to reproduce the bug?

weixiong-ur commented 1 week ago

Hi, I faced with the same issue using the latest diffusers and FSDP. I wonder if you have resolved this issue?