Error in tutorial notebook LLama3-8b

[riccardo-unipg]

Why in the following prompt of LLama 3-8b tutorial notebook (https://colab.research.google.com/drive/135ced7oHytdxu3N2DNe1Z0kqjyYIkDXp?usp=sharing):

_alpaca_prompt = """Below is an instruction that describes a task, paired with an input that provides further context. Write a response that appropriately completes the request.

### Instruction: {}

### Input: {}

### Response: {}"""

EOS_TOKEN = tokenizer.eos_token # Must add EOS_TOKEN def formatting_prompts_func(examples): instructions = examples["instruction"] inputs = examples["input"] outputs = examples["output"] texts = [] for instruction, input, output in zip(instructions, inputs, outputs):

Must add EOS_TOKEN, otherwise your generation will go on forever!

    text = alpaca_prompt.format(instruction, input, output) + EOS_TOKEN
    texts.append(text)
return { "text" : texts, }

pass

from datasets import load_dataset dataset = load_dataset("yahma/alpaca-cleaned", split = "train") dataset = dataset.map(formatting_promptsfunc, batched = True,)

, you put the value of the Response? I mean, it doesnt' be empty to permit the model to generate the Response? Something like: ### Response: """ and pass only: text = alpaca_prompt.format(instruction, input) + EOS_TOKEN.

[danielhanchen]

So for finetuning you have to train on the responses - only for inference you need to leave the response field empty (so the LLM generates it)

[riccardo-unipg]

So for finetuning you have to train on the responses - only for inference you need to leave the response field empty (so the LLM generates it)

Thank you.