Notebook - SciPy2024/module1/1-introduction-to-language-models.ipynb
Function - text_generator
next word is chosen based on below code:
word = model_sorted_probabilities[
np.random.choice(np.arange(len(model_sorted_probabilities)))
][0]
Should be something like this instead:
words = [item[0] for item in model_sorted_probabilities] # Get list of words
probabilities = [item[1] for item in model_sorted_probabilities] # Get corresponding probabilities
word = np.random.choice(words, p=probabilities) # Select based on probabilities
Notebook - SciPy2024/module1/1-introduction-to-language-models.ipynb Function - text_generator
next word is chosen based on below code:
word = model_sorted_probabilities[ np.random.choice(np.arange(len(model_sorted_probabilities))) ][0]
Should be something like this instead:
words = [item[0] for item in model_sorted_probabilities] # Get list of words probabilities = [item[1] for item in model_sorted_probabilities] # Get corresponding probabilities word = np.random.choice(words, p=probabilities) # Select based on probabilities