Update: I tried some Gaussian like function as activations, some resulted well. If the key is the bell-shape, then I guess anything that gives out the bell shape can work also well enough.
Original comment:
If you see this, comment anything to let me know. Thanks.
I saw the activation, it's interesting. I tried this
x = np.linspace(-5, 5, 201)
sig = 3#2#1#0.1#0.3#0.5
l = 6#4#2
numerator = -np.power(x, ll)
denominator = 2sig*sig
result = np.exp(numerator/denominator)
fig, ax = plt.subplots(1)
ax.plot(x, result)
If this is exactly what in your code, then I guess the "result*cos(x)" can result better.
Good luck.
Update: I tried some Gaussian like function as activations, some resulted well. If the key is the bell-shape, then I guess anything that gives out the bell shape can work also well enough.
Original comment: If you see this, comment anything to let me know. Thanks. I saw the activation, it's interesting. I tried this x = np.linspace(-5, 5, 201) sig = 3#2#1#0.1#0.3#0.5 l = 6#4#2
numerator = -np.power(x, ll) denominator = 2sig*sig result = np.exp(numerator/denominator)
fig, ax = plt.subplots(1) ax.plot(x, result)
If this is exactly what in your code, then I guess the "result*cos(x)" can result better. Good luck.