issues
search
dllllb
/
pytorch-lifestream
A library built upon PyTorch for building embeddings on discrete event sequences using self-supervision
Apache License 2.0
215
stars
46
forks
source link
Batchnorm in NN Head
#157
Open
ivan-chai
opened
4 months ago
ivan-chai
commented
4 months ago
There are two problems with nn.Head:
Linear layer when followed by BatchNorm must not include a trainable bias vector (bias=False).
Batchnorm needs a tensor with shape (B, C, L), while PTLS uses (B, L, C). The conversion between these two formats must be added to the forward method.
There are two problems with nn.Head: