search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.88k stars 232 forks source link

Errors when using MEND with gpt2-xl #232

Closed ksasi closed 6 months ago

ksasi commented 6 months ago

On Executing the following code block , after setting the parameters for MEND

editor=BaseEditor.from_hparams(hparams)
metrics, edited_model, _ = editor.edit(
    prompts=prompts,
    ground_truth=None,
    target_new=target_new,
    subject=subject,
    keep_original_weight=False
)
print(metrics)
print(type(edited_model))

we are seeing the following error :

2024-04-17 08:56:32,112 - easyeditor.editors.editor - INFO - Instantiating model 2024-04-17 08:56:32,112 - easyeditor.editors.editor - INFO - Instantiating model 2024-04-17 08:56:32,112 - easyeditor.editors.editor - INFO - Instantiating model 2024-04-17 08:56:32,112 - easyeditor.editors.editor - INFO - Instantiating model 2024-04-17 08:56:32,112 - easyeditor.editors.editor - INFO - Instantiating model 04/17/2024 08:56:32 - INFO - easyeditor.editors.editor - Instantiating model 2024-04-17 08:56:50,873 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left... 2024-04-17 08:56:50,873 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left... 2024-04-17 08:56:50,873 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left... 2024-04-17 08:56:50,873 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left... 2024-04-17 08:56:50,873 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left... 04/17/2024 08:56:50 - INFO - easyeditor.editors.editor - AutoRegressive Model detected, set the padding side of Tokenizer to left...

RuntimeError Traceback (most recent call last) Cell In[14], line 1 ----> 1 editor=BaseEditor.from_hparams(hparams) 2 metrics, editedmodel, = editor.edit( 3 prompts=prompts, 4 ground_truth=None, (...) 7 keep_original_weight=False 8 ) 9 print(metrics)

File ~/model-edit/EasyEdit/easyeditor/editors/editor.py:59, in BaseEditor.from_hparams(cls, hparams) 56 @classmethod 57 def from_hparams(cls, hparams: HyperParams): ---> 59 return cls(hparams)

File ~/model-edit/EasyEdit/easyeditor/editors/editor.py:126, in BaseEditor.init(self, hparams) 124 hparams.device = str(self.model.device).split(":")[1] 125 if not hparams.model_parallel and hasattr(hparams, 'device'): --> 126 self.model.to(f'cuda:{hparams.device}') 128 self.hparams = hparams

File /opt/conda/envs/medit/lib/python3.9/site-packages/transformers/modeling_utils.py:1902, in PreTrainedModel.to(self, *args, *kwargs) 1897 raise ValueError( 1898 ".to is not supported for 4-bit or 8-bit models. Please use the model as it is, since the" 1899 " model has already been set to the correct devices and casted to the correct dtype." 1900 ) 1901 else: -> 1902 return super().to(args, **kwargs)

File /opt/conda/envs/medit/lib/python3.9/site-packages/torch/nn/modules/module.py:1145, in Module.to(self, *args, **kwargs) 1141 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, 1142 non_blocking, memory_format=convert_to_format) 1143 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) -> 1145 return self._apply(convert)

File /opt/conda/envs/medit/lib/python3.9/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn) 795 def _apply(self, fn): 796 for module in self.children(): --> 797 module._apply(fn) 799 def compute_should_use_set_data(tensor, tensor_applied): 800 if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): 801 # If the new tensor has compatible tensor type as the existing tensor, 802 # the current behavior is to change the tensor in-place using .data =, (...) 807 # global flag to let the user control whether they want the future 808 # behavior of overwriting the existing tensor or not.

File /opt/conda/envs/medit/lib/python3.9/site-packages/torch/nn/modules/module.py:797, in Module._apply(self, fn) 795 def _apply(self, fn): 796 for module in self.children(): --> 797 module._apply(fn) 799 def compute_should_use_set_data(tensor, tensor_applied): 800 if torch._has_compatible_shallow_copy_type(tensor, tensor_applied): 801 # If the new tensor has compatible tensor type as the existing tensor, 802 # the current behavior is to change the tensor in-place using .data =, (...) 807 # global flag to let the user control whether they want the future 808 # behavior of overwriting the existing tensor or not.

File /opt/conda/envs/medit/lib/python3.9/site-packages/torch/nn/modules/module.py:820, in Module._apply(self, fn) 816 # Tensors stored in modules are graph leaves, and we don't want to 817 # track autograd history of param_applied, so we have to use 818 # with torch.no_grad(): 819 with torch.no_grad(): --> 820 param_applied = fn(param) 821 should_use_set_data = compute_should_use_set_data(param, param_applied) 822 if should_use_set_data:

File /opt/conda/envs/medit/lib/python3.9/site-packages/torch/nn/modules/module.py:1143, in Module.to..convert(t) 1140 if convert_to_format is not None and t.dim() in (4, 5): 1141 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, 1142 non_blocking, memory_format=convert_to_format) -> 1143 return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)

RuntimeError: CUDA error: invalid device ordinal Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.

Please suggest any alternative.

XeeKee commented 6 months ago

This issue doesn't seem to be related to the easyedit code. I think it might be due to your CUDA setup not being configured correctly. You can check if your CUDA settings use .to("cuda:n") and make sure that n does not exceed the GPU index available on your machine.

zxlzr commented 6 months ago

Hi, have you solved your issue yet?

ksasi commented 6 months ago

Thanks for the suggestions, I will check this and will let you know.

EmilyGirl commented 5 months ago

How can I solve this? It's throwing an error during runtime.

CUDA error: device-side assert triggered Compile with TORCH_USE_CUDA_DSA to enable device-side assertions. File "/media/sev/Linux/wym/easyEdit/easyeditor/evaluate/evaluate.py", line 310, in icl_lm_eval logits = model(input_ids=input_ids, attention_mask=attention_mask).logits File "/media/sev/Linux/wym/easyEdit/easyeditor/evaluate/evaluate.py", line 199, in compute_icl_edit_quality edit_acc = icl_lm_eval(model, model_name, hparams, tok, icl_examples, File "/media/sev/Linux/wym/easyEdit/easyeditor/editors/editor.py", line 259, in edit "post": compute_icl_edit_quality(self.model, self.model_name, self.hparams, self.tok, icl_examples, File "/media/sev/Linux/wym/easyEdit/examples/run_knowedit_llama2.py", line 207, in metrics, editedmodel, = editor.edit( RuntimeError: CUDA error: device-side assert triggered Compile with TORCH_USE_CUDA_DSA to enable device-side assertions.