search

stanford-futuredata / ARES

Automated Evaluation of RAG Systems
https://ares-ai.vercel.app/
Apache License 2.0
486 stars 53 forks source link

Error while training classifier #68

Open islambs231bi opened 3 months ago

islambs231bi commented 3 months ago 
classifier_model {
        training_dataset: ["./aresdocs/synthetic_queries_1.tsv"] 
        validation_set: ["./aresdocs/nq_labeled_output.tsv"] 
        label_column: ["Context_Relevance_Label"]
        num_epochs: 10 
        model_choice: google-t5/t5-small
        patience_value: 3 
        learning_rate: 0.000005
        assigned_batch_size: 1
        gradient_accumulation_multiplier: 32  
}

Error while training google-t5/t5-small with error:

  File "/home/islam/.local/lib/python3.10/site-packages/ares/ares.py", line 137, in train_classifier
    binary_classifer_config(**self.classifier_model_config)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/binary_classifier.py", line 164, in binary_classifer_config
    model, avg_train_losses, avg_valid_losses, eval_dataloader, inference_times = train_and_evaluate_model(train_and_eval_settings)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/LLM_as_a_Judge_Adaptation/General_Binary_Classifier.py", line 793, in train_and_evaluate_model
    outputs = model(**new_batch)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/LLM_as_a_Judge_Adaptation/General_Binary_Classifier.py", line 221, in forward
    total_output = self.encoderModel(ids, attention_mask=mask)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1517, in forward
    decoder_outputs = self.decoder(
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1010, in forward
    raise ValueError(f"You have to specify either {err_msg_prefix}input_ids or {err_msg_prefix}inputs_embeds")
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds