Update 08-seq_classification.ipynb

[Gaaaavin]

Made changes to the line 507 and 508 of 08-seq_classification.ipynb. The original lines were incorrect, making the evaluation of model incorrect.

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB

[Atcold]

Can you provide some context, plz?

[Gaaaavin]

In the original code, we use

sequence_end = torch.tensor([len(sequence) for sequence in data_decoded]) - 1
output = output[torch.arange(data.shape[0]).long(), sequence_end, :]

to get the output classification from the model. In other words, we treat the len(sequence)-th element in the output as the classification. However, this is not how we trained the model. During training, we always treat the last element in the output as the classification:

# Pick only the output corresponding to last sequence element (input is pre padded)
output = output[:, -1, :]

I think this is the reason why the accuracy in the model evaluation block is very low and doesn't correspond to the log of training.

[Gaaaavin]

Btw, I've also posed a thread on Campuswire: https://campuswire.com/c/GCEF8E4E7/feed/82

[Atcold]

I see what's going on… https://github.com/Atcold/pytorch-Deep-Learning/pull/19 was only partially correcting this issue. Very good, @Gaaaavin, you got extra credit for this!

[JonathanSum]

@Gaaaavin Is it possible to share the thread on Campuswire to let the people in public learn about it? Thank you.

I guess the point of this post is len(sequence) is wrong because it was the last element from the data or treating the "last element" as the classification output is how the training part uses.

[Gaaaavin]

In the original code, we use

sequence_end = torch.tensor([len(sequence) for sequence in data_decoded]) - 1
output = output[torch.arange(data.shape[0]).long(), sequence_end, :]

to get the output classification from the model. In other words, we treat the len(sequence)-th element in the output as the classification. However, this is not how we trained the model. During training, we always treat the last element in the output as the classification:

# Pick only the output corresponding to last sequence element (input is pre padded)
output = output[:, -1, :]

I think this is the reason why the accuracy in the model evaluation block is very low and doesn't correspond to the log of training.

Basically it's the same as this one