search

1ytic / warp-rnnt

CUDA-Warp RNN-Transducer
MIT License
211 stars 41 forks source link

Implement FastEmit regularization #23

Closed iceychris closed 2 years ago

iceychris commented 2 years ago

Hey!

Thank you for this great library!

This PR implements FastEmit regularization from https://arxiv.org/abs/2010.11148. The gradients of all non-blank symbols are scaled up by a small factor (around 1.004). Intuitively, the model is encouraged to output symbols faster, thus reducing latency.

image

1ytic commented 2 years ago

Thank you! Looks simple. Unfortunately I will not be able to test the final streaming performance, but I will happily add this code. Could you fix core_gather.cu as well? This is the most efficient implementation and should be used by default.

iceychris commented 2 years ago

Nice, thank you!