NN ML 范式

[mgttt]

神经网络 / 机器学习 / 范式

• 输入训练张量：样本X、结果Y
• 超参组H，如神经元数目、损失、激活、映射、层次和结构等，为NNOP所用
• 变参组T，自H初始化并应用于NNOP，一般如权张量、偏置张量等
• 模型主操作NNOP计算输出张量P=NNOP(T,X,H)
• 损失L，最常用最小二乘法，即欧距平方，也有其他损失函数
• 炼参：一般是H条件下求L最小时的T
• 模型保存：（T、H、NN）

e.g. 分类回归 Logistic Regression: A=(x)=>1/(1+EXP(-x)) PROD=DOT H=[A,PROD,NN] NNOPX=([W1,B1,W2,B2],X,[A,PROD])=>(PROD(A(PROD(X,W1)-B1),W2)-B2) NNOP=(T,X,H)=>NNOPX(UPK(T),X,UPK(H)) MSE=(y,p)=>MEAN((y-p)**2) LOSS_NN=(T,X,H,Y)=>MSE(NNOP(T,X,H),Y) MINIMIZE(LOSS_NN,INIT(H),arg=(X,H,Y))

• 好的超参和网络NNOP要有较好的拟合度和泛化度。
• “NNOP”模型结构，比如前后向反馈，函数化时按时间帧同步。模型其实就是结构➕参数，以后探索全异步

1/(1+EXP(-x)): (E(-X)+1)(-1), 1/(1+1/exp(x))，傅立叶变换与自然对数底e的惯性要联系起来

[mgttt]

e.g. in python

import numpy as np
from scipy.optimize import minimize
from numba import njit
##############################

nn = 9
T = np.random.rand(3*nn+1)
c_sample = 128
x = np.linspace(start=-10,stop=10,num=c_sample).reshape(-1,1) # vectorized
y_real = np.sin(x)/x
y_sample = np.sin(x)/x + (np.random.rand(c_sample,1)*0.1-0.05)

X = x
Y = y_sample
H = np.array([nn])

##############################

PROD = np.dot
EXP = np.exp
MEAN = np.mean

@njit
def A(x): return 1/(1+EXP(-x))
MSE = lambda Y,P: MEAN((Y-P)**2)

@njit
def UPK(T,H):
    NN = H[0]
    W1= T[:NN].reshape(1, NN)
    W2= T[NN:NN*2].reshape(NN, 1)
    B1= T[NN*2:-1].reshape(-1, NN)
    B2= T[-1]
    return W1,W2,B1,B2

@njit
def NNOP(T,X,H):
    W1,W2,B1,B2 = UPK(T,H)
    return PROD(A(PROD(X,W1)-B1),W2)-B2

LOSS_NN = lambda T,X,H,Y: MSE(NNOP(T,X,H),Y)

SOLVE = lambda T,X,Y,H:minimize(LOSS_NN,T,(X,H,Y)).x

RS = SOLVE(T,X,Y,H)

##############################

print('MODEL TO SAVE:\n',UPK(RS,H))

import matplotlib.pyplot as plt
x_test = np.sort(np.random.rand(100,1)*10-5,axis=0)
y_test_real = np.sin(x_test)/x_test
y_test_pred = NNOP(RS, x_test, H)
plt.plot(x_test, y_test_real, 'oy')
plt.plot(x_test, y_test_pred, '-b')
plt.legend(['y_test_real', 'y_test_pred'])
#plt.title('mse={0}'.format(MSE(y_test_real,y_test_pred)))
plt.show()