FelixBenning / optimization-review

0 stars 0 forks source link

Justify eigenvectors are standard basis with High Dimensionality? #11

Open FelixBenning opened 2 years ago

FelixBenning commented 2 years ago

In high dimension random vectors are likely orthogonal (cf. https://math.stackexchange.com/questions/995623/why-are-randomly-drawn-vectors-nearly-perpendicular-in-high-dimensions/4307960#4307960)

Can this justify ada-heuristics? I.e. orthogonal to most except for a small subset which has likely a different eigenvalue distribution within than all the eigenvalues? E.g. two dimensional: it is likely that a random vector has more of one standard basis component than the other. So if the standard basis vectors where the eigenvectors, this random vector would have more of one eigenvalue than the other in some sense. So the ada heuristics would capture some of that probably?

Try on quadratic functions