search

karpathy / makemore

An autoregressive character-level language model for making more things
MIT License
2.47k stars 652 forks source link

Fix bug in RNN where hprev always referred to start. #3

Closed normanyu closed 2 years ago

normanyu commented 2 years ago

Change so that hprev refers to output of previous cell.

As a sanity check, I ran the code with and without the fix and compared it to the Bigram baseline model.

python makemore.py -i names.txt -o names --max-steps=10000 --type=rnn

fixed_rnn_makemore

karpathy commented 2 years ago

Yes of course, thank you!