ml5js / ml5-library

Friendly machine learning for the web! 🤖
https://ml5js.org
Other
6.46k stars 899 forks source link

[charRnn] Off-by-one error in charRNN.feed #778

Open averyhiebert opened 4 years ago

averyhiebert commented 4 years ago

→ Step 1: Describe the issue 📝

There is an off-by-one error in CharRNN's "feed" function. The first character of the string to be fed is read twice, and the last character is ignored (i.e. if you try to feed the string "hello" to the model, what the model actual sees is "hhell").

This can be clearly seen by mentally stepping through lines 282-297 of CharRNN/index.js, paying close attention to the indices. The fix also seems clear (changing the i in line 296 to i + 1). However, I've also included a demonstration of the incorrect behaviour in practice.

→ Step 2: Screenshots or Relevant Documentation 🖼

n/a

→ Step 3: Share an example of the issue 🦄

Consider the following function, which seeds the model with a given string, and then predicts the next character.

rnn = ml5.charRNN('models/bolano/',function(){
    console.log("Model loaded");
});

function test_prediction(input_string){
    rnn.reset();
    rnn.feed(input_string,function(){
        rnn.predict(0.5).then(function(content){
            console.log("Prediction: '" + content["sample"] + "'");
            console.log("Probability of a space is: " 
                + prob_of(content.probabilities," "));
        })
    })
}

After the seed phrase "Where is the", we can make an educated guess that the model should predict a space with high probability. Instead, we get the following output (or something similar):

>> test_prediction("Where is the");
Prediction: 'e'
Probability of a space is: 2.2018098421030608e-15

If we add any arbitrary character to the end of the seed string, then we get the expected prediction of a space, even if it no longer makes sense.

>> test_prediction("Where is thei");
Prediction: ' '
Probability of a space is: 0.9992843866348267

This demonstration is kinda unnecessary, though, since the error is clear when inspecting the code.

→ Describe your setup 🦄

I'm using Linux Mint 18.3, Firefox 64.0, and version ml5 0.4.3, not that it's relevant in this case.

lindapaiste commented 2 years ago

Thank you for this very well-documented issue! I'm sorry that it has taken so long to get fixed, but it is resolved by #1355