search

akshaygopalkr / EM-VLM4AD

MIT License
36 stars 0 forks source link

Error in notebook train_T5_Base #2

Open munirfarzeen opened 1 month ago

munirfarzeen commented 1 month ago

Hi,

I am using train_T5_Base notebook and getting this error when using the model gpt2. COuld you help, how to fix this error?

3027 if size_average is not None or reduce is not None: 3028 reduction = _Reduction.legacy_get_string(size_average, reduce) -> 3029 return torch._C._nn.cross_entropy_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index, label_smoothing)

ValueError: Expected input batch_size (1032) to match target batch_size (180).

akshaygopalkr commented 1 month ago

Hello! For now changing the LLM to GPT-2 is not supported. I think the reason you are getting this error is because the GPT model expects the inputs and labels to be the exact same to calculate the loss. Since this code repository supports T5 which is an encoder-decoder model, the inputs will the text for the questions and the labels will be the answers.