uw-ssec / tutorials

SSEC tutorials for various topics
https://uw-ssec-tutorials.readthedocs.io
BSD 3-Clause "New" or "Revised" License
4 stars 5 forks source link

fix: Text generator function in Scipy tutorial not sampling words based on calculated probabilities #138

Open apoorvasheera opened 1 month ago

apoorvasheera commented 1 month ago

Notebook - SciPy2024/module1/1-introduction-to-language-models.ipynb Function - text_generator

next word is chosen based on below code:

word = model_sorted_probabilities[ np.random.choice(np.arange(len(model_sorted_probabilities))) ][0]

Should be something like this instead:

words = [item[0] for item in model_sorted_probabilities] # Get list of words probabilities = [item[1] for item in model_sorted_probabilities] # Get corresponding probabilities word = np.random.choice(words, p=probabilities) # Select based on probabilities