search

okrasolar / pytorch-timeseries

PyTorch implementations of neural networks for timeseries classification
107 stars 23 forks source link

Different implementation for ResNet #4

Open JAEarly opened 1 year ago

JAEarly commented 1 year ago

I've noticed some differences to the original ResNet code at https://github.com/cauchyturing/UCR_Time_Series_Classification_Deep_Learning_Baseline/blob/master/ResNet.py.

You're missing at BatchNorm right at the start of the network, and also the residual should be added before applying ReLU in the final layer of each ResNet block, not after applying ReLU.

In my experiments, adding the above two changes seems to generate results that better match the original results.

JAEarly commented 1 year ago

Just to add a bit more to this, I don't think the BatchNorm right at the start is necessary if the data is z-normalised, but adding the residual before applying ReLU definitely improves performance.