[Deprecated] Size mismatch when setting knowledge_usage to separate

[puraminy]

For the evaluation, I receive the following error:

...
  File "/home/pouramini/UnifiedSKG/utils/trainer.py", line 298, in prediction_step
    **gen_kwargs,
  File "/home/pouramini/UnifiedSKG/models/unified/prefixtuning.py", line 289, in generate
    bsz=bsz, sample_size=kwargs['num_beams'], description=description_representation, knowledge=knowledge_representation,
  File "/home/pouramini/UnifiedSKG/models/unified/prefixtuning.py", line 120, in get_prompt
    past_key_values = torch.cat([past_key_values, self.knowledge_trans(knowledge)], dim=1)
RuntimeError: Sizes of tensors must match except in dimension 0. Got 4 and 16 (The offending index is 0)

It seems when you use num_beam greater than 1 and it's used as sample_size in get_prompt method and is multiplied to batch_size, the mismatch occurs.

    def get_prompt(self, bsz=None, sample_size=1, description=None, knowledge=None):
        old_bsz = bsz
        bsz = bsz * sample_size
        input_tokens = self.input_tokens.unsqueeze(0).expand(bsz, -1)
        temp_control = self.wte(input_tokens)
        if description is not None:
            temp_control = temp_control + description.repeat_interleave(sample_size, dim=0).unsqueeze(1)
        past_key_values = self.control_trans(temp_control)  # bsz, seqlen, layer*emb
        if knowledge is not None:
            past_key_values = torch.cat([past_key_values, self.knowledge_trans(knowledge)], dim=1)

[ChenWu98]

Hi,

Yes, another bug when setting knowledge_usage to separate. I tried to fix it in the latest commit.

[puraminy]

I applied the changes in commit, but I get the error in training:

1779, in training_step
    loss = self.compute_loss(model, inputs)
  File "/home/pouramini/anaconda3/lib/python3.9/site-packages/transformers/trainer.py", line 1811, in compute_loss
    outputs = model(**inputs)
  File "/home/pouramini/anaconda3/lib/python3.9/site-packages/torch/nn/modules/module.py", line 1102, in _call_impl
    return forward_call(*input, **kwargs)
  File "/home/pouramini/mt5-comet/comet/models/unified/prefixtuning.py", line 268, in forward
    past_prompt = self.get_prompt(
  File "/home/pouramini/mt5-comet/comet/models/unified/prefixtuning.py", line 123, in get_prompt
    past_key_values = torch.cat([past_key_values, self.knowledge_trans(knowledge.repeat_interleave(sample_size, dim=0).unsqueeze(1))], dim=1)
RuntimeError: Tensors must have same number of dimensions: got 3 and 4

Did you try it yourself?

[puraminy]

Thanks, now it works, I also removed unsqueeze part

[ChenWu98]

Great!

[puraminy]

Thanks, now it's working!

[ChenWu98]

Yea I just ran it on my server, and it also works. Thanks for pointing out this issue!