search

zjunlp / EasyEdit

[ACL 2024] An Easy-to-use Knowledge Editing Framework for LLMs.
https://zjunlp.github.io/project/KnowEdit
MIT License
1.91k stars 235 forks source link

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! when trying EasyEdit_Example_IKE.ipynb #196

Closed asdfo123 closed 8 months ago

asdfo123 commented 8 months ago

Hi, I correctly run the code when I use single GPU environment (Autodl), But when I switch to multi_GPUs environment (4*3090), this RuntimeError occured. My hparams:

alg_name: "IKE"
model_name: "./gpt2-xl"
sentence_model_name: "./all-MiniLM-L6-v2"
device: 1
results_dir: "./results"

k: 32
model_parallel: false

I'm trying to use the 2nd GPU independtly in order not to disturb other Processes. What should I modify? This is the error information:

RuntimeError                              Traceback (most recent call last)
Cell In[6], [line 80](vscode-notebook-cell:?execution_count=6&line=80)
     [78](vscode-notebook-cell:?execution_count=6&line=78) edited_batch = tokenizer(ike_generation_prompts, return_tensors='pt', padding=True, max_length=max_length)
     [79](vscode-notebook-cell:?execution_count=6&line=79) batch = tokenizer(generation_prompts, return_tensors='pt', padding=True, max_length=max_length)
---> [80](vscode-notebook-cell:?execution_count=6&line=80) pre_edit_outputs = model.generate(
     [81](vscode-notebook-cell:?execution_count=6&line=81)     input_ids=batch['input_ids'].to('cuda'),
     [82](vscode-notebook-cell:?execution_count=6&line=82)     attention_mask=batch['attention_mask'].to('cuda:1'),
     [83](vscode-notebook-cell:?execution_count=6&line=83)     max_length=max_length
     [84](vscode-notebook-cell:?execution_count=6&line=84) )
     [85](vscode-notebook-cell:?execution_count=6&line=85) post_edit_outputs = edited_model.generate(
     [86](vscode-notebook-cell:?execution_count=6&line=86)     input_ids=edited_batch['input_ids'].to('cuda:1'),
     [87](vscode-notebook-cell:?execution_count=6&line=87)     attention_mask=edited_batch['attention_mask'].to('cuda:1'),
     [88](vscode-notebook-cell:?execution_count=6&line=88)     max_length=max_length
     [89](vscode-notebook-cell:?execution_count=6&line=89) )
     [90](vscode-notebook-cell:?execution_count=6&line=90) print('*'*100)

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:115](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:115), in context_decorator.<locals>.decorate_context(*args, **kwargs)
    [112](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:112) @functools.wraps(func)
    [113](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:113) def decorate_context(*args, **kwargs):
    [114](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:114)     with ctx_factory():
--> [115](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/utils/_contextlib.py:115)         return func(*args, **kwargs)

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1522](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1522), in GenerationMixin.generate(self, inputs, generation_config, logits_processor, stopping_criteria, prefix_allowed_tokens_fn, synced_gpus, assistant_model, streamer, **kwargs)
   [1516](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1516)         raise ValueError(
   [1517](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1517)             "num_return_sequences has to be 1 when doing greedy search, "
   [1518](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1518)             f"but is {generation_config.num_return_sequences}."
   [1519](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1519)         )
   [1521](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1521)     # 11. run greedy search
-> [1522](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1522)     return self.greedy_search(
   [1523](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1523)         input_ids,
   [1524](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1524)         logits_processor=logits_processor,
   [1525](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1525)         stopping_criteria=stopping_criteria,
   [1526](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1526)         pad_token_id=generation_config.pad_token_id,
   [1527](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1527)         eos_token_id=generation_config.eos_token_id,
   [1528](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1528)         output_scores=generation_config.output_scores,
   [1529](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1529)         return_dict_in_generate=generation_config.return_dict_in_generate,
   [1530](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1530)         synced_gpus=synced_gpus,
   [1531](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1531)         streamer=streamer,
   [1532](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1532)         **model_kwargs,
   [1533](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1533)     )
   [1535](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1535) elif is_contrastive_search_gen_mode:
   [1536](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:1536)     if generation_config.num_return_sequences > 1:

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2339](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2339), in GenerationMixin.greedy_search(self, input_ids, logits_processor, stopping_criteria, max_length, pad_token_id, eos_token_id, output_attentions, output_hidden_states, output_scores, return_dict_in_generate, synced_gpus, streamer, **model_kwargs)
   [2336](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2336) model_inputs = self.prepare_inputs_for_generation(input_ids, **model_kwargs)
   [2338](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2338) # forward pass to get next token
-> [2339](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2339) outputs = self(
   [2340](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2340)     **model_inputs,
   [2341](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2341)     return_dict=True,
   [2342](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2342)     output_attentions=output_attentions,
   [2343](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2343)     output_hidden_states=output_hidden_states,
   [2344](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2344) )
   [2346](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2346) if synced_gpus and this_peer_finished:
   [2347](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/generation/utils.py:2347)     continue  # don't waste resources running the code we don't need

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   [1496](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1496) # If we don't have any hooks, we want to skip the rest of the logic in
   [1497](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1497) # this function, and just call forward.
   [1498](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1498) if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   [1499](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1499)         or _global_backward_pre_hooks or _global_backward_hooks
   [1500](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1500)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501)     return forward_call(*args, **kwargs)
   [1502](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1502) # Do not call functions when jit is used
   [1503](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1503) full_backward_hooks, non_full_backward_hooks = [], []

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1080](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1080), in GPT2LMHeadModel.forward(self, input_ids, past_key_values, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, labels, use_cache, output_attentions, output_hidden_states, return_dict)
   [1072](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1072) r"""
   [1073](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1073) labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
   [1074](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1074)     Labels for language modeling. Note that the labels **are shifted** inside the model, i.e. you can set
   [1075](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1075)     `labels = input_ids` Indices are selected in `[-100, 0, ..., config.vocab_size]` All labels set to `-100`
   [1076](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1076)     are ignored (masked), the loss is only computed for labels in `[0, ..., config.vocab_size]`
   [1077](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1077) """
   [1078](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1078) return_dict = return_dict if return_dict is not None else self.config.use_return_dict
-> [1080](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1080) transformer_outputs = self.transformer(
   [1081](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1081)     input_ids,
   [1082](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1082)     past_key_values=past_key_values,
   [1083](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1083)     attention_mask=attention_mask,
   [1084](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1084)     token_type_ids=token_type_ids,
   [1085](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1085)     position_ids=position_ids,
   [1086](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1086)     head_mask=head_mask,
   [1087](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1087)     inputs_embeds=inputs_embeds,
   [1088](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1088)     encoder_hidden_states=encoder_hidden_states,
   [1089](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1089)     encoder_attention_mask=encoder_attention_mask,
   [1090](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1090)     use_cache=use_cache,
   [1091](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1091)     output_attentions=output_attentions,
   [1092](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1092)     output_hidden_states=output_hidden_states,
   [1093](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1093)     return_dict=return_dict,
   [1094](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1094) )
   [1095](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1095) hidden_states = transformer_outputs[0]
   [1097](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:1097) # Set device for model parallelism

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   [1496](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1496) # If we don't have any hooks, we want to skip the rest of the logic in
   [1497](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1497) # this function, and just call forward.
   [1498](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1498) if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   [1499](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1499)         or _global_backward_pre_hooks or _global_backward_hooks
   [1500](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1500)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501)     return forward_call(*args, **kwargs)
   [1502](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1502) # Do not call functions when jit is used
   [1503](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1503) full_backward_hooks, non_full_backward_hooks = [], []

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:846](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:846), in GPT2Model.forward(self, input_ids, past_key_values, attention_mask, token_type_ids, position_ids, head_mask, inputs_embeds, encoder_hidden_states, encoder_attention_mask, use_cache, output_attentions, output_hidden_states, return_dict)
    [843](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:843) head_mask = self.get_head_mask(head_mask, self.config.n_layer)
    [845](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:845) if inputs_embeds is None:
--> [846](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:846)     inputs_embeds = self.wte(input_ids)
    [847](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:847) position_embeds = self.wpe(position_ids)
    [848](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/transformers/models/gpt2/modeling_gpt2.py:848) hidden_states = inputs_embeds + position_embeds

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501), in Module._call_impl(self, *args, **kwargs)
   [1496](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1496) # If we don't have any hooks, we want to skip the rest of the logic in
   [1497](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1497) # this function, and just call forward.
   [1498](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1498) if not (self._backward_hooks or self._backward_pre_hooks or self._forward_hooks or self._forward_pre_hooks
   [1499](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1499)         or _global_backward_pre_hooks or _global_backward_hooks
   [1500](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1500)         or _global_forward_hooks or _global_forward_pre_hooks):
-> [1501](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1501)     return forward_call(*args, **kwargs)
   [1502](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1502) # Do not call functions when jit is used
   [1503](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/module.py:1503) full_backward_hooks, non_full_backward_hooks = [], []

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:162](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:162), in Embedding.forward(self, input)
    [161](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:161) def forward(self, input: Tensor) -> Tensor:
--> [162](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:162)     return F.embedding(
    [163](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:163)         input, self.weight, self.padding_idx, self.max_norm,
    [164](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/modules/sparse.py:164)         self.norm_type, self.scale_grad_by_freq, self.sparse)

File [~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2210](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2210), in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
   [2204](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2204)     # Note [embedding_renorm set_grad_enabled]
   [2205](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2205)     # XXX: equivalent to
   [2206](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2206)     # with torch.no_grad():
   [2207](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2207)     #   torch.embedding_renorm_
   [2208](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2208)     # remove once script supports set_grad_enabled
   [2209](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2209)     _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
-> [2210](https://vscode-remote+ssh-002dremote-002b172-002e26-002e94-002e14.vscode-resource.vscode-cdn.net/data/edit/EasyEdit/tutorial-notebooks/~/anaconda3/envs/EasyEdit/lib/python3.9/site-packages/torch/nn/functional.py:2210) return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:1 and cuda:0! (when checking argument for argument index in method wrapper_CUDA__index_select)

This is information in run.log:

2024-03-06 15:52:58,392 - easyeditor.editors.editor - INFO - Instantiating model
2024-03-06 15:52:58,392 - easyeditor.editors.editor - INFO - Instantiating model
2024-03-06 15:52:58,392 - easyeditor.editors.editor - INFO - Instantiating model
2024-03-06 15:52:58,392 - easyeditor.editors.editor - INFO - Instantiating model
2024-03-06 15:53:09,753 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left...
2024-03-06 15:53:09,753 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left...
2024-03-06 15:53:09,753 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left...
2024-03-06 15:53:09,753 - easyeditor.editors.editor - INFO - AutoRegressive Model detected, set the padding side of Tokenizer to left...
2024-03-06 15:53:11,989 - easyeditor.editors.editor - INFO - Execution 0 editing took 0.37169790267944336
2024-03-06 15:53:11,989 - easyeditor.editors.editor - INFO - Execution 0 editing took 0.37169790267944336
2024-03-06 15:53:11,989 - easyeditor.editors.editor - INFO - Execution 0 editing took 0.37169790267944336
2024-03-06 15:53:11,989 - easyeditor.editors.editor - INFO - Execution 0 editing took 0.37169790267944336
2024-03-06 15:53:12,195 - easyeditor.editors.editor - INFO - Evaluation took 0.20032215118408203
2024-03-06 15:53:12,195 - easyeditor.editors.editor - INFO - Evaluation took 0.20032215118408203
2024-03-06 15:53:12,195 - easyeditor.editors.editor - INFO - Evaluation took 0.20032215118408203
2024-03-06 15:53:12,195 - easyeditor.editors.editor - INFO - Evaluation took 0.20032215118408203
2024-03-06 15:53:12,201 - easyeditor.editors.editor - INFO - 0 editing: Q: The president of the US is? A: -> Joe Biden  
 {'pre': {'rewrite_acc': 0.5, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'time': 0.37169790267944336, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}
2024-03-06 15:53:12,201 - easyeditor.editors.editor - INFO - 0 editing: Q: The president of the US is? A: -> Joe Biden  
 {'pre': {'rewrite_acc': 0.5, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'time': 0.37169790267944336, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}
2024-03-06 15:53:12,201 - easyeditor.editors.editor - INFO - 0 editing: Q: The president of the US is? A: -> Joe Biden  
 {'pre': {'rewrite_acc': 0.5, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'time': 0.37169790267944336, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}
2024-03-06 15:53:12,201 - easyeditor.editors.editor - INFO - 0 editing: Q: The president of the US is? A: -> Joe Biden  
 {'pre': {'rewrite_acc': 0.5, 'portability': {}, 'rephrase_acc': 1.0}, 'case_id': 0, 'requested_rewrite': {'prompt': 'Q: The president of the US is? A:', 'target_new': 'Joe Biden', 'ground_truth': 'Donald Trump', 'portability': {}, 'locality': {}, 'subject': 'president', 'rephrase_prompt': 'The leader of the United State is'}, 'time': 0.37169790267944336, 'post': {'rewrite_acc': 1.0, 'locality': {}, 'portability': {}, 'rephrase_acc': 1.0}}

By the way, what should I do when I want to use multi-GPUs for method requiring large VRAMs?

XeeKee commented 8 months ago

You can try setting the CUDA_VISIBLE_DEVICES, this may help you.

asdfo123 commented 8 months ago

You can try setting the CUDA_VISIBLE_DEVICES, this may help you.

Thanks, that helps me a lot!