search

adepierre / caffe-char-rnn

Multi-layer Recurrent Neural Networks (with LSTM) for character-level language models in Caffe
26 stars 7 forks source link

Strange output #4

Closed nzinger16 closed 6 years ago

nzinger16 commented 6 years ago

Hi,

I was able to compile and run your caff-char-rnn design but it does not match the output you have in the README. I tried giving different seeds and temperatures but it always reverts back to the same sentence it repeats. See below:

_To be, or not to be: that is the question:

I will not see the state of the world is a state, And therefore shall be so for the state, And therefore shall be so for the state of the world in the state of the world of the world of the world and the state of the world of the world of the world, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state, And the world is a stranger of the state,_

nzinger16 commented 6 years ago

Also thank you for sharing this design example! It was very helpful for one of my projects.

adepierre commented 6 years ago

I can reproduce the output you give when I set the temperature to 0 (so the character with the highest probability is selected with a probability of 1). So it's kind of normal to rapidly have some repetitions. With higher temperatures, I have more variability.

My guess is that the command line you launch is not correct and the temperature flag is not read correctly, so it is always set to the default value of 0.0. Here is the command line I use:

"../bin/Release/caffe_char_rnn" --train=false --sequence_length=75 --batch_size=25 --temperature=0.5 --vocabulary=vocabulary_62.txt --model=caffe_char_rnn.prototxt --weights=caffe_char_rnn_iter_100000.caffemodel --seed="To be, or not to be: that is the question:" --number_of_predictions=1500 --output_file=test.txt

nzinger16 commented 6 years ago

You're right in the sense that the temperature was 0 but I gave it a value of 0.5 in my script. I changed your code below (line 75) of Classifier.cpp and it worked:

temperature = abs(temperature); -> temperature = std::abs(temperature); Depending on which version of gcc/c++ the user has, the absolute function is overloaded and cast into an int which will always be 0 for any value less than 1.

You might want to change it in case someone else runs into this issue :).

Thank you for the quick reply!

adepierre commented 6 years ago

Thanks for the fix!