Regardless of class output fiaxtions are the same

[joeyearsley]

I hard coded the output points to be: [[900], [900], [900], [900], [900]]

and received exactly the same points as if I had hard coded: [[100], [100], [100], [100], [100]]

Is this a fluke or did you see similar results?

The only reason I can think of is due to the original activations matching the learned filters hence firing more, providing a majority of the activations regardless of the output. Whilst the output helps guide the remaining points to areas of interest.

[mopurikreddy]

We have observed similar behaviour too, it is not a fluke. As you suspected, we can explain this behaviour with the distributed nature of the learned representations (e.g. 4096D feature in fc7 of Alexnet, similarly other features like fc6, etc.). These activations encode input information across multiple neurons (among the available 4096 neurons in case of fc7) in the same layer and it is not possible to separate the neurons into semantic groups. That is, it is not possible to identify the neurons that fire only for a specific visual stimulus (say dog images). Because of this, the strongly fired neurons are not different for different categories (during backtracking). Therefore we end up tracing the same path onto the object regions regardless of the label.

[insikk]

@mopurikreddy I want to get the pixel level evidences for feed forward pass without specifying particular label. If label change does not change much of the visualization input, is there a correct way not to give initial values for points at prediction layer?