hi, thanks to your peoject, i re-implement conformer for tf.1.15. When i train it with ctc loss on 1000-h chinese audio dataset, the loss is fluctuating and could not decline.
the conv-sampling is 3-layer conv2d with 144 filters and kernel_size=3, reduction_factor=4
the conformer is just like the bert-base but with relative position encoding using t-5
the optimizer is adam-with-weight-decay, params are default as bert-base
.
could you help me with this?
hi, thanks to your peoject, i re-implement conformer for tf.1.15. When i train it with ctc loss on 1000-h chinese audio dataset, the loss is fluctuating and could not decline. the conv-sampling is 3-layer conv2d with 144 filters and kernel_size=3, reduction_factor=4 the conformer is just like the bert-base but with relative position encoding using t-5 the optimizer is adam-with-weight-decay, params are default as bert-base . could you help me with this?