merge with anatome

[brando90]

https://github.com/moskomule/anatome/pull/11/files

[brando90]

tests work:

/Users/brando/anaconda3/envs/metalearning/bin/python /Applications/PyCharm.app/Contents/plugins/python/helpers/pydev/pydevd.py --cmd-line --multiproc --qt-support=auto --client 127.0.0.1 --port 55429 --file /Users/brando/ultimate-utils/tutorials_for_myself/anatome_pg/sanity_checks_anatome.py
Connected to pydev debugger (build 212.5080.64)
--- Sanity check: sCCA = 1.0 when using same net twice with same input. --
Should be very very close to 1.0: sim=0.9999997615814209 (cxa_dist_type='svcca')
Is it close to 1.0? True
Should be very very close to 1.0: sim=1.0 (cxa_dist_type='pwcca')
Is it close to 1.0? True
Should be very very close to 1.0: sim=1.0 (cxa_dist_type='lincka')
Is it close to 1.0? True
Should be very very close to 1.0: sim=0.9998071789741516 (cxa_dist_type='opd')
Is it close to 1.0? True
--- Sanity check: when number of data points B is smaller than D, then it should be trivial to make similiarty 1.0 (even if nets/matrices are different)
Should be very very close to 1.0: sim=1.0000001192092896 (since we have many features to match the two Xw1, Yw2).
Is it close to 1.0? True
-- Santity: just makes sure that when low data is present sim is high and afterwards (as n->infty) sim (CCA) converges to the "true" cca value (eventually)
sims=[1.0, 1.0, 0.9808495044708252, 0.9202049374580383, 0.628921627998352, 0.384316623210907, 0.2671157121658325, 0.18725836277008057, 0.11984008550643921, 0.08449167013168335]
Should be very very close to 1.0: sim=1.0000001192092896
Is it close to 1.0? True
sims=[0.2887918949127197, 0.42893028259277344, 0.6236045360565186, 0.9157285094261169, 0.9184483885765076, 0.9743918776512146, 0.9904040694236755, 0.9901362061500549, 0.9911755919456482, 0.9917169809341431, 0.9912896156311035]
import sys; print('Python %s on %s' % (sys.version, sys.platform))

[brando90]

but in anatome was the implementation of the normalization:

(input - input.mean(dim=dim, keepdim=True)) / norm(input, 'fro')

missing parenthesis.

[brando90]

unsure if this will happen...