search

Haiyang-W / TokenFormer

Official Implementation of TokenFormer: Rethinking Transformer Scaling with Tokenized Model Parameters
https://haiyang-w.github.io/tokenformer.github.io/
Apache License 2.0
277 stars 16 forks source link

Fails with last version of pytorch #6

Open kroggen opened 1 week ago

kroggen commented 1 week ago

The message:

python3 deepy.py generate.py -d configs tokenformer/1-5B_eval.yml text_generation.yml --text_gen_type interactive

[2024-11-07 00:21:17,970] [INFO] [real_accelerator.py:161:get_accelerator] Setting ds_accelerator to cuda (auto detect)
/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/zero/linear.py:47: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @autocast_custom_fwd
/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/zero/linear.py:66: FutureWarning: `torch.cuda.amp.custom_bwd(args...)` is deprecated. Please use `torch.amp.custom_bwd(args..., device_type='cuda')` instead.
  @autocast_custom_bwd
Traceback (most recent call last):
  File "/root/TokenFormer/deepy.py", line 19, in <module>
    import deepspeed.launcher.runner
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/__init__.py", line 22, in <module>
    from . import module_inject
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/module_inject/__init__.py", line 6, in <module>
    from .replace_module import replace_transformer_layer, revert_transformer_layer, ReplaceWithTensorSlicing, GroupQuantizer, generic_injection
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/module_inject/replace_module.py", line 587, in <module>
    from ..pipe import PipelineModule
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/pipe/__init__.py", line 6, in <module>
    from ..runtime.pipe import PipelineModule, LayerSpec, TiedLayerSpec
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/pipe/__init__.py", line 6, in <module>
    from .module import PipelineModule, LayerSpec, TiedLayerSpec
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/pipe/module.py", line 19, in <module>
    from ..activation_checkpointing import checkpointing
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/activation_checkpointing/checkpointing.py", line 26, in <module>
    from deepspeed.runtime.config import DeepSpeedConfig
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/config.py", line 41, in <module>
    from ..elasticity import (
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/elasticity/__init__.py", line 10, in <module>
    from .elastic_agent import DSElasticAgent
  File "/opt/conda/lib/python3.11/site-packages/deepspeed/elasticity/elastic_agent.py", line 9, in <module>
    from torch.distributed.elastic.agent.server.api import log, _get_socket_with_port
ImportError: cannot import name 'log' from 'torch.distributed.elastic.agent.server.api' (/opt/conda/lib/python3.11/site-packages/torch/distributed/elastic/agent/server/api.py)

This error occurs because there's an incompatibility between the installed versions of PyTorch and DeepSpeed. The log function was removed from torch.distributed.elastic.agent.server.api in a newer version of PyTorch, but DeepSpeed is still trying to import it

Haiyang-W commented 2 days ago

Pytorch 2.2 is worked now. I havn't checked the last version.