Open chiragjn opened 1 year ago
cc @alaradirik and @amyeroberts
Hi @chiragjn, I was able to replicate the error on my local (also macOS-13.1-x86_64-i386-64bit) and I'm looking into the issue.
A quick update - I tracked down the issue to the accelerate library, setting device_map=True
sets low_cpu_mem_usage
to True. This causes the model parameters to be initialized as meta tensors, which can not be copied to CPU or GPU without tensor conversion.
This issue also affects DETA, Conditional DETR, Deformable DETR and Table Transformers as they have identical frozen modules that are initialized by copying the parameters of their respective backbone models. We will be opening a fix PR shortly!
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.
Please note that issues that do not follow the contributing guidelines are likely to be ignored.
Hey, is there any progress with this issue?
Hi @AlonZolfi, @alaradirik has now left Hugging Face, so I'm picking this up.
As @alaradirik mentions, this arises as a consequence the replacement of the batch norm in the backbone of these models. I'll be digging into it properly next week when I have a bit more time.
Re-opening the issue as it's not yet solved and will keep you posted!
It closed again, there was some progress with the issue?
@amyeroberts the problem is indeed annoying, I have similar problem fine-tuning some models like llama. anyone working to solve it?
Hey @amyeroberts, was this issue solved already?
@AlonZolfi @ranchlai No, I unfortunately haven't had bandwidth to address this yet. I'm marking it as a difficult issue that anyone in the community can try and tackle if they wish.
I got the same problem when using accelerate, doing model.cuda()
worked as expected.
The related PR is #26150
where:
from transformers import AutoModelForSeq2SeqLM
model = AutoModelForSeq2SeqLM.from_pretrained(model_name, device_map="auto")
so pinging @muellerzr as this is probably related to our hf hooks. Now I might be creating the buffer and tensors in a wrong way but can´t get it to load so help is appreciated! (See the UMTRelativePositionalBias
class)
(using accelerate 0.22.0)
cc @SunMarc
Hi @ArthurZucker , I left a few comments on the PR to explain the issue. Hope that you have enough context to fix the problem ;)
System Info
transformers
version: 4.28.1Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
This does not work because the
transformers.models.detr.modeling_detr.DetrConvEncoder
model init involves copy weights fromnn.BatchNorm2d
toDetrFrozenBatchNorm2d
which is not allowed when on a meta device.The model loads fine with a specific device with
device
argument.