Closed georgemavrakis-wings closed 3 years ago
Thank you! We'll fix it asap
Hello, I have done tests and get the correct results. But I have noticed that in the tensors you have sent, the result of the eddl is the equivalent of applying the SELU twice to the input. Could it be that you are applying the operation twice?
Hello,
No, I have the Selu layer initialized only once. I have constructed the following network:
Xin_t = Tensor.fromarray(Xin) # Xin: Selu_input.npy array
in_ = eddl.Input([128, Xin_t.shape[-1]])
layer = in_
layer2 = eddl.Selu(parent=layer)
layer3 = eddl.GlobalAveragePool1D(parent=layer2)
layer4 = eddl.Squeeze(parent=layer3)
out = eddl.Constant(l=eddl.Dense(parent=layer4, ndim=2), v=0.03)
net = eddl.Model(in_=[in_], out=[out])
eddl.build(net, eddl.adam(0.001), ["soft_cross_entropy"], ["categorical_accuracy"], eddl.CS_CPU())
eddl.train_batch(net, [Xin_t], [y_train], indices=np.linspace(0, 9, 10, dtype=int).tolist()) # y_train: (10x2) label array
Then, I use the following code to take the output of the Selu layer from the above net:
OUT = net.layers[1].output
OUT = OUT.getdata()
The result is equal with the pyeddl_SELU_output.npy array, which is different from the pytorch_SELU_output.npy.
Ok, your are right. There is a bug in the backward function of the Selu. In the inital tests that I made I used the predict function so I didn't see the error because the backward is not executed. Now it is fixed. Thank you!
Good afternoon,
I have a problem with the Selu layer in Pyeddl. I have opened an issue in the pyeddl repo, but I was redirected here.
More specifically, it seems that Selu layer does not calculate its output correctly. I have tested other layers instead (e.g. Elu, ReLu) and the output is as expected. All layers have been compared with PyTorch and seem to produce the same output, except for Selu.
I provided the input and output numpy arrays of the selu in pyeddl and pytorch in the above pyeddl repo issue. I use Linux OS, with the latest version of pyeddl 0.13.0 (EDDL 0.9.1b).
Thank you in advance.