namisan / mt-dnn

Multi-Task Deep Neural Networks for Natural Language Understanding
MIT License
2.23k stars 411 forks source link

It is difficult to understand Figure 1 in the SMART paper. #220

Open yupeijei1997 opened 3 years ago

yupeijei1997 commented 3 years ago

Figure a in Figure 1 shows the decision boundary without Rs, and Figure b shows the decision boundary with Rs.But it gives me the feeling that the decision boundary of Figure a is smoother, while the decision boundary of Figure b is a bit over-fitting. In the paper, you want to use Figure 1 to show that after the disturbance is added, the output f of the neural network will not change too much, but the figure also gives the decision boundary, and the amount of change is whether to add Rs, Rs itself It's not a disturbance, but a regular term, so it makes me feel very confused, can you explain it?

Finally, my feeling is that in Figure a, the Rs regular term is not added, but the decision boundary is smoother, that is, the generalization performance is better; Figure b, the Rs regular term is added, but it is over-fitting, and the generalization performance Deterioration; and the added disturbance and the output f of the neural network are not shown on the graph;

In addition, it is mentioned in Figure 1 that the output f will not change too much in the vicinity of the training data, but this sentence is not visible on the figure;

Want to understand what you mean, please explain, thank you!