EchteRobert
commented
2 years ago Noise-scaling experiment
As outlined by Mehrtash:
With regards to the noise scaling -- here's what I had in mind. It's not a super formal idea, and probably not a super useful idea haha. Consider an aggregation model which operates as follows: it ignores low-saliency cells, and it averages the representation of high-saliency cells. Let's say the representation of i'th high-saliency cell, zi, is concentrated around a single point z0 in the repr. space along with a bit of Gaussian noise to account for cell-to-cell variation, i.e. zi = z0 + εi, where ϵi ∼ N(0, σ). If you feed this aggregation model n high siliency cells and any number of low-saliency cells, it will ignore the latter and return the average of the former, i.e. zaggr = z0 + (1/n) ∑i=1n εi = z0 + εaggr, where ϵaggr ∼ N(0, σ/√n). So if you plot the aggregated representation (or its PCs) vs.the number of high-saliency cells n, you would expect to get a local "jitter" that goes down like . Now, consider a smarter aggregation model -- it identifies your high-saliency cells, "snaps" them to z0 and returns that, like k-means. Powerful enough neural networks can learn to do such things. Max pooling layers and ReLU activation help achieve this type of behavior faster, but there's no theoretical guarantee whatsoever. In any event -- such a smart aggregation model will output z0 if you feed it any n > 0 high-saliency cells, and shrink the noise all the way to zero. The output is constant z0. I don't believe that's how your model is working at all given that the network output is not stationary. I don't believe your model is filtering bad cells and averaging over high-saliency cells either, again, because the output is not stationary -- it is doing something else :-) If you wish to take a deeper look at noise scaling of your model output out of curiosity, you could do that by windowing the admixing fractions around each point, regressing out the linear trend, and calculating the standard deviation of the residual. You could then make a log-log plot of the noise std vs. admixing fraction and assess whether it looks like any power law. Curiously, the noise seems to be constant all the way up to 0.6 and then it rapidly shrinks. Not sure what to make of it :-)
In this issue, I will outline experiments that may be useful in the future to further investigate the inner-workings of the model, but for which I currently do not have the time (or have low priority).