dhaivat1729
commented
4 months ago No, you are mixing training and inference. Here is the explanation:
1. Training vs. Inference:
- Training Phase: During the training of a language model like a large language model (LLM), the model learns by predicting the next word based on the previous words in the input data. However, it's important to understand that the model does not use its own predictions to continue the training sequence.
- Inference Phase: In contrast, during inference (or generation), the model uses the words it has just predicted to generate subsequent words. This is when the model can potentially "go down the wrong path" if it makes a wrong prediction, as each new word depends on the previously generated words.
2. How Training Works:
- Input Data: During training, you provide the model with sequences of text from your dataset.
- Ground Truth: For each sequence, the model is shown part of the sequence (e.g., the first few words) and asked to predict the next word. Importantly, whether the prediction is correct or not, the model is then given the actual next word (the "ground truth") for its next prediction. This process helps the model learn the correct associations without compounding errors from its own predictions.
Example:
Imagine you're training a model on the phrase "The quick brown fox jumps over the lazy dog."
Training Process:
- Input to Model: "The quick brown fox jumps"
- Correct Next Word (Ground Truth): "over"
- Model's Prediction: Suppose the model predicts "around" instead of "over."
- Feedback: The model is informed that the correct word was "over." It does not continue with "around" for the next training step.
- Next Input: Regardless of the model’s incorrect prediction, the next input during training will still be "The quick brown fox jumps over" (to predict "the"), not "The quick brown fox jumps around."
This process ensures that the model is consistently trained on correct sequences, helping it learn the right patterns without the errors in its predictions affecting its training.
mostafamdy
Feature request
Hi,
I have a misunderstanding regarding training LLMs. When we train the model, we calculate the loss by having the model predict the next word and then compute the difference between the true and predicted values.
What I want to know is: when making the model predict the next words, it generates a new word based on the previously generated words. If a generated word is wrong, won't the subsequent predictions continue down the wrong path?
Motivation
Your contribution
.