rgeirhos
commented
5 days ago Hi @MLDeS thanks for your interest.
The example is called a toy example since we distill the problem down to a simple setting (2 classes, stars and moons). The full example can be reproduced (including dataset, network architecture, training etc.) using the code from this github repo - see https://github.com/rgeirhos/shortcut-perspective/blob/master/code/shortcut_toy_exp.py.
As we discuss in the paper, the definition of intended/unintended features is human-centric. For instance, sometimes it may be intended that a network relies on the image background for classification, sometimes it may be unintended.
Does this address your questions?
Hello everyone, thanks for the nice work, reading the paper is enjoyable!
I have a question on Figure 2. This says it is a toy example, I assume, this is not run on real data but only a visualization, or? The reason I ask this because, the figure exemplifies that a neural network in the classification of moon vs. star relies on location rather than shape, and this is unintended. Is that what the neural network would really learn, given enough data?
An extended question is, how do we know when an unintended learning occurs? It is only after probing the model after training? And, in a general case, how do we distinguish between what a model is learning is really a case of unintended learning? Or just another way of learning? Like it is another useful feature for learning? Some learning in our perspective can be unintended since we wanted the network/system to learn a certain reasoning, but yet it may be a useful learning for the example.
What I mean is, is it rather our task definition, than network learning unintendedly that causes shortcut learning?
Thanks a lot!