Operatornorm for Changing Learning rate

Basis change matrices do not cancel out

Ideas

[ ] Piecewise constant learning rate?
[ ] Basis change matrices are maybe similar enough, add basis change for iteration n and bound the difference between basis change n+1 and change n
- [ ] somehow surgically remove the basis change differences from the product without forcing a product of operator norms