search

stanford-futuredata / ARES

Automated Evaluation of RAG Systems
https://ares-ai.vercel.app/
Apache License 2.0
486 stars 53 forks source link

Training classifiers error #63

Open Islam231bi opened 4 months ago

Islam231bi commented 4 months ago 
from ares import ARES

classifier_config = {
    "training_dataset": ["./documents/synthetic_queries_1.csv"], 
    "validation_set": ["./documents/nq_labeled_output.tsv"], 
    "label_column": ["Context_Relevance_Label"], 
    "num_epochs": 10, 
    "patience_value": 3, 
    "learning_rate": 5e-6,
    "assigned_batch_size": 1,
    "model_choice": "t5-small",  
    "gradient_accumulation_multiplier": 32,  
}

ares = ARES(classifier_model=classifier_config)
ares.train_classifier()
Traceback (most recent call last):
  File "/home/islam/Projects/brightskies/repos/BSRAG/test_ares.py", line 16, in <module>
    ares.train_classifier()
  File "/home/islam/.local/lib/python3.10/site-packages/ares/ares.py", line 137, in train_classifier
    binary_classifer_config(**self.classifier_model_config)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/binary_classifier.py", line 164, in binary_classifer_config
    model, avg_train_losses, avg_valid_losses, eval_dataloader, inference_times = train_and_evaluate_model(train_and_eval_settings)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/LLM_as_a_Judge_Adaptation/General_Binary_Classifier.py", line 793, in train_and_evaluate_model
    outputs = model(**new_batch)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/ares/LLM_as_a_Judge_Adaptation/General_Binary_Classifier.py", line 217, in forward
    total_output = self.encoderModel(input_ids=ids, attention_mask=mask)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1517, in forward
    decoder_outputs = self.decoder(
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
    return forward_call(*args, **kwargs)
  File "/home/islam/.local/lib/python3.10/site-packages/transformers/models/t5/modeling_t5.py", line 1010, in forward
    raise ValueError(f"You have to specify either {err_msg_prefix}input_ids or {err_msg_prefix}inputs_embeds")
ValueError: You have to specify either decoder_input_ids or decoder_inputs_embeds

I get this error when trying to train a classifier, I installed ares using pip