The paper from which the picture in question 11 of the midterm was taken

[noranta4]

The picture in question 11 was taken from the paper "Learning explanations that are hard to vary" where it is discussed immediately in the introduction.

We assigned full points to two types of answers in the midterm:

• Answers arguing that the minima bottom left is better than the one top right in the total loss, so answers that follow the same view of the authors.
• Answers that effectively and extensively describes what is shown in the image. In particular two aspects should be noted:
  • The red paths are iterations of GD or SGD from different initialization of the parameters.
  • The non-point-like minima in the losses for data A and data B become a point-like minimum in the total loss, just like the minimum bottom left.