search

vgel / repeng

A library for making RepE control vectors
https://vgel.me/posts/representation-engineering/
MIT License
435 stars 31 forks source link

Error when running honesty.ipynb #16

Closed hojmax closed 4 months ago

hojmax commented 4 months ago

When i run honesty.ipynb i get the following error when instantiating the controlmodel:


Cell In [2], [line 8](vscode-notebook-cell:?execution_count=2&line=8)
      [6](vscode-notebook-cell:?execution_count=2&line=6) model = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=torch.float16)
      [7](vscode-notebook-cell:?execution_count=2&line=7) model = model.to("cuda:0" if torch.cuda.is_available() else "mps:0" if torch.backends.mps.is_available() else "cpu")
----> [8](vscode-notebook-cell:?execution_count=2&line=8) model = ControlModel(model, list(range(-5, -18, -1)))
     [10](vscode-notebook-cell:?execution_count=2&line=10) user_tag, asst_tag = "[INST]", "[/INST]"

File [/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:30](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:30), in ControlModel.__init__(self, model, layer_ids)
     [27](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:27) super().__init__()
     [28](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:28) self.model = model
---> [30](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:30) layers = model_layer_list(model)
     [31](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:31) self.layer_ids = [i if i >= 0 else len(layers) + i for i in layer_ids]
     [32](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:32) for layer_id in layer_ids:

File [/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:207](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:207), in model_layer_list(model)
    [204](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:204)     model = model.model
    [206](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:206) if hasattr(model, "model"):  # mistral-like
--> [207](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:207)     return model.layers
    [208](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:208) elif hasattr(model, "transformer"):  # gpt-2-like
    [209](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/repeng/control.py:209)     return model.transformer.h

File [/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1688](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1688), in Module.__getattr__(self, name)
   [1686](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1686)     if name in modules:
   [1687](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1687)         return modules[name]
-> [1688](https://file+.vscode-resource.vscode-cdn.net/opt/homebrew/lib/python3.11/site-packages/torch/nn/modules/module.py:1688) raise AttributeError(f"'{type(self).__name__}' object has no attribute '{name}'")

AttributeError: 'MistralForCausalLM' object has no attribute 'layers'```