Error when loading models to DirectML after PR#24505

[pengan1987]

System Info

• transformers version: 4.31.0
• Platform: Windows-10-10.0.19045-SP0
• Python version: 3.10.13
• Huggingface_hub version: 0.17.3
• Safetensors version: 0.3.3
• Accelerate version: 0.23.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.0.0+cpu (False)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?:
• Using distributed or parallel set-up in script?:

Who can help?

@sgu

Information

• [ ] The official example scripts
• [X] My own modified scripts

Tasks

• [ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• [X] My own task or dataset (give details below)

Reproduction

I'm trying to run Phi-1.5 using Pytorch with DirectML torch-directml==0.2.0.dev230426, my script is here: https://github.com/pengan1987/DirectML-demos/blob/main/Phi_1_5.py When transformers<=4.30.2, it works fine, when transformers>=4.31.0, I got an exception like this when loading the model It seems modeling_utils.py has been changed a lot in PR#24505 and caused this exception, is there any fix/workarond to address this issue?

E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\torch\utils\_device.py:62: UserWarning: TypedStorage is deprecated. It will be removed in the future and Unty
pedStorage will be the only storage class. This should only matter to you if you are using storages directly.  To access UntypedStorage directly, use tensor.untyped_sto
rage() instead of tensor.storage()
  return func(*args, **kwargs)
Traceback (most recent call last):
  File "E:\StableDiffusion\DirectML-demos\Phi_1_5.py", line 14, in <module>
    model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\models\auto\auto_factory.py", line 488, in from_pretrained
    return model_class.from_pretrained(
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\modeling_utils.py", line 2903, in from_pretrained
    ) = cls._load_pretrained_model(
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\modeling_utils.py", line 3061, in _load_pretrained_model
    id_tensor = id_tensor_storage(tensor) if tensor.device != torch.device("meta") else id(tensor)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\transformers\pytorch_utils.py", line 287, in id_tensor_storage
    return tensor.device, storage_ptr(tensor), storage_size(tensor)
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\safetensors\torch.py", line 33, in storage_size
    return tensor.untyped_storage().nbytes()
  File "E:\StableDiffusion\miniconda3\envs\pydml\lib\site-packages\torch\utils\_device.py", line 62, in __torch_function__
    return func(*args, **kwargs)
NotImplementedError: Cannot access storage of OpaqueTensorImpl

Expected behavior

model loading correctly like transformers<=4.30.2

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[amyeroberts]

Hi @pengan1987, thanks for raising this issue!

Two comments:

• There have been a few recent changes to loading of weights in recent releases. Could you try with the latest version of transformers to see if that works?
• Could you provide a minimal code snippet which reproduces the error? Without being able to get the error on our side we won't be able to help

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.