AttributeError: 'FalconAttention' object has no attribute 'maybe_rotary'

OpenGVLab / OmniQuant

[ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

MIT License

626 stars 49 forks source link

AttributeError: 'FalconAttention' object has no attribute 'maybe_rotary' #71

Open luchangli03 opened 3 months ago

luchangli03 commented 3 months ago

OmniQuant-main/models/int_falcon_layer.py", line 52, in init self.maybe_rotary = copy.deepcopy(org_module.maybe_rotary) File "local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1614, in getattr raise AttributeError("'{}' object has no attribute '{}'".format( AttributeError: 'FalconAttention' object has no attribute 'maybe_rotary'

transformers version: 4.39.3

chaibosong commented 3 weeks ago

I encountered the same issue. It seems that the transformers package does not have maybe_rotary. How did you solve this problem later?I tried different versions of the transformers package.

transformers version: 4.32.0