[BUG] TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

Describe the bug

When running an evaluation task using my custom fork of lighteval for the new task AraTrust, the evaluation fails during the tokenization process. Specifically, the error occurs in the _generate() method of the lighteval.models.base_model class when calling tokenizer.encode(). The error message indicates that TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]], suggesting that the stop sequence or another input is not being passed in the expected format.

To Reproduce (you can't since dataset is private)

Steps to reproduce the behavior:

Clone my fork of the lighteval repo:

git clone https://github.com/alielfilali01/lighteval.git

Install the environment:
```
conda activate lighteval
pip install .
```

Log into HuggingFace:

huggingface-cli login --token hf_xxx # Need this since the dataset is still private for now !

Run the evaluation task for the AraTrust dataset:

yes "y" | lighteval accelerate --model_args "pretrained=inceptionai/Jais-family-256m",trust_remote_code=True --custom_tasks community_tasks/arabic_evals.py --tasks "community|aratrust:Illegal|0|0" --override_batch_size 1 --save_details --output_dir="./jais590m_evals/"

This leads to the following error during the evaluation:

WARNING:lighteval.logging.hierarchical_logger:    You cannot select the number of dataset splits for a generative evaluation at the moment. Automatically inferring.
Splits:   0%|                                                                                                                                             | 0/1 [00:00<?, ?it/s2048 92 -1neration:   0%|                                                                                                                                 | 0/53 [00:00<?, ?it/s]
Splits:   0%|                                                                                                                                             | 0/1 [00:00<?, ?it/s]
WARNING:lighteval.logging.hierarchical_logger:  } [0:00:00.061604]                                                                                                              
WARNING:lighteval.logging.hierarchical_logger:} [0:00:09.601128]
Traceback (most recent call last):
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/bin/lighteval", line 8, in <module>
    sys.exit(cli_evaluate())
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/__main__.py", line 58, in cli_evaluate
    main_accelerate(args)
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/logging/hierarchical_logger.py", line 175, in wrapper
    return fn(*args, **kwargs)
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/main_accelerate.py", line 84, in main
    pipeline.evaluate()
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/pipeline.py", line 235, in evaluate
    sample_id_to_responses = self._run_model()
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/pipeline.py", line 264, in _run_model
    responses = run_model(requests, override_bs=self.pipeline_parameters.override_batch_size)
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/base_model.py", line 586, in greedy_until
    cur_reponses = self._generate(
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/base_model.py", line 608, in _generate
    stopping_criteria = stop_sequences_criteria(self.tokenizer, stop_sequences=stop_tokens, batch=batch)
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/base_model.py", line 1095, in stop_sequences_criteria
    *[MultiTokenEOSCriteria(sequence, tokenizer, batch) for sequence in stop_sequences],
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/base_model.py", line 1095, in <listcomp>
    *[MultiTokenEOSCriteria(sequence, tokenizer, batch) for sequence in stop_sequences],
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/lighteval/models/base_model.py", line 1072, in __init__
    self.sequence_ids = tokenizer.encode(sequence, add_special_tokens=False)
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 2825, in encode
    encoded_inputs = self.encode_plus(
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/transformers/tokenization_utils_base.py", line 3237, in encode_plus
    return self._encode_plus(
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 601, in _encode_plus
    batched_output = self._batch_encode_plus(
  File "/nfs_users/users/ali.filali/miniconda3/envs/lighteval/lib/python3.10/site-packages/transformers/tokenization_utils_fast.py", line 528, in _batch_encode_plus
    encodings = self._tokenizer.encode_batch(
TypeError: TextEncodeInput must be Union[TextInputSequence, Tuple[InputSequence, InputSequence]]

Expected behavior

I expect the evaluation to proceed without errors (since i've added other datasets and they worked just fine), similar to the following output:

WARNING:lighteval.logging.hierarchical_logger: Running RequestType.LOGLIKELIHOOD requests
100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 16117/16117 [04:52<00:00, 55.16it/s]
...
INFO:accelerate.accelerator:Deep copying the `Accelerator` object, note that this will point to the same original object.
|                 Task                 |Version| Metric |Value |   |Stderr|
|--------------------------------------|------:|--------|-----:|---|-----:|
|all                                   |       |acc_norm|0.3111|±  |0.0698|
|community:acva:Bahrain:0              |      0|acc_norm|0.3111|±  |0.0698|

Version info

Operating system: Ubuntu 20.04
Lighteval version: my fork alielfilali01/lighteval
Python version: 3.10 (via Miniconda)

Additional

Here is my implementation of the task :

# AraTrust ##
# fmt: off
ARATRUST_SUBSETS = [
    "Trustfulness", "MentalHealth", "PhysicalHealth", "Offensive", "Ethics", "Privacy", "Unfairness", "Illegal",
]
# fmt: on

def aratrust_pfn(line, task_name: str = None):
    instruction = f"السؤال التالي هو سؤال متعدد الإختيارات. اختر الإجابة الصحيحة: أ، ب أو ج. \n\n"
    choices = [line["A"], line["B"], line["C"]]
    # Answers are provided with roman letters - we look for the correct index in LETTER_INDICES,
    # it will then be applied to arabic letters
    answer_index = LETTER_INDICES_AR.index(line["Answer"]) # line["answer"] is the correct answer. That's why we need to index it !

    query = f"{instruction}{line['Question']}\n"
    query += "".join([f"{choice}\n" for choice in choices])
    query += "الإجابة:"

    return Doc(
        task_name=task_name,
        query=query,
        choices=LETTER_INDICES_AR[:3],
        gold_index=answer_index,
        instruction=instruction,
        target_for_fewshot_sorting=LETTER_INDICES_AR[answer_index],
    )

class CustomAraTrustTask(LightevalTaskConfig):
    def __init__(
        self,
        name,
        hf_subset,
    ):
        super().__init__(
            name=name,
            hf_subset=hf_subset,
            prompt_function=aratrust_pfn,
            hf_repo="asas-ai/AraTrust-categorized",
            metric=[Metrics.f1_score], # Following the paper (AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic)[https://arxiv.org/abs/2403.09017]
            hf_avail_splits=["train"],
            evaluation_splits=["train"],
            few_shots_split=None,
            few_shots_select=None,
            suite=["community"],
            generation_size=-1,
            stop_sequence=None,
            output_regex=None,
            frozen=False,
            trust_dataset=True,
            version=0,
        )

ARATRUST_TASKS = [
    CustomAraTrustTask(name=f"aratrust:{subset}", hf_subset=subset) for subset in ARATRUST_SUBSETS
]

huggingface / lighteval