Discrepancy in makemore_part1_bigrams single sample draw

karpathy / nn-zero-to-hero

Neural Networks: Zero to Hero

MIT License

10.9k stars 1.33k forks source link

Discrepancy in makemore_part1_bigrams single sample draw #34

Closed anoopelias closed 1 year ago

anoopelias commented 1 year ago

When I run the makemore_part1_bigrams notebook, in the cell where we draw a single sample based on probability distribution in the first row of N, I get a different sample ( 'c' ) compared to the one in the video ( 'm' ). Everything else until then seems to be the same, with manually seeded generator, I'd expect even this to match exactly.

Value in this repo (and the video):

My run:

I have pushed the full run until this cell in the commit here.

What am I missing?

ratelle commented 1 year ago

This seems to be a bug in pytorch 2.0.0 when calling multinomial with num_samples=1. If you run the same code using pytorch 1.13.1, you will get the correct result.

anoopelias commented 1 year ago

Thank you! This solves the problem. Thread to the original discussion here.