There is no reproductivity on PyTorch==1.4.0!!

syoamakase / ASR

Speech Recognition Toolkit

8 stars 0 forks source link

There is no reproductivity on PyTorch==1.4.0!! #5

Open syoamakase opened 4 years ago

syoamakase commented 4 years ago

In some examples, we cannot confirm the same results of README.md. I think that batch size, pytorch version, and single or multi gpu(s) are relevant to the reproductivity.

Tedlium 2:

batch_size = 30, pytorch==1.4.0, and single gpu: bad results (WER is about 40%)
batch_size = 40, pytorch==1.4.0, and multi gpus: extremely bad results (the same outputs)

Please report if you cannot confirm the reproductivity.

syoamakase commented 4 years ago

I confirmed that the performances of Tedlium2 were good in PyTorch's version 1.3.0 and 1.5.1 when I used both single gpu (batch size = 30) and multi gpus (batch size = 40).

I consider the PyTorch version is relevant to this bad results. I highly recommend to avoid pytorch==1.4.0!!!!