open-connectome-classes / StatConn-Spring-2015-Info

introductory material
18 stars 4 forks source link

Sparsity vs. Loss/Risk (Matlab from last class) #141

Open dlee138 opened 9 years ago

dlee138 commented 9 years ago

From the matlab code we wrote last class, what was the conclusion we had on sparsity vs. value of loss & risk. Does sparser = lower risk? It was a bit unclear to me how it worked since the code was bugging out towards the end of class and it would be helpful if anyone could elaborate.

yaxigeigei commented 9 years ago

Sparcity = low risk, yes, but only by the specific way we calculated it. The criterion Jovo choosed was for delibrately reducing the risk. We can choose other criteria that have the same or opposite effects. Therefore, how to choose a reasonable/reliable/objective loss function is something he left us to think - what makes sense?

jovo commented 9 years ago

to be a little more concrete, sparse refers to a statistical distribution, low risk refers to the performance of a particular decision rule on a particular distribution.

when the distribution yields samples that are sparse, a decision rule that imposes sparsity correctly can indeed reduce risk, but it can increase risk, as there are lots of notions of sparsity.

the point is that the interplay between the distribution and the decision rule is key for determining risk.

On Wed, Mar 4, 2015 at 10:47 PM, Duo Xu notifications@github.com wrote:

Sparcity = low risk, yes, but only by the specific way we calculated it. The criterion Jovo choosed was for delibrately reducing the risk. We can choose other criteria that have the same or opposite effects. Therefore, how to choose a reasonable/reliable/objective loss function is something he left us to think - what makes sense?

— Reply to this email directly or view it on GitHub https://github.com/Statistical-Connectomics-Sp15/intro/issues/141#issuecomment-77301552 .

the glass is all full: half water, half air. openconnecto.me, we're hiring! https://docs.google.com/document/d/14SApYAzxF0Ddqg2ZCEwjmz3ht2TDhDmxyZI2ZP82_0U/edit?usp=sharing , jovo.me, my calendar https://www.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York