Why do we need to compile the tokens to FST in wsj recipe?

srvk / eesen

The official repository of the Eesen project

http://arxiv.org/abs/1507.08240

Apache License 2.0

824 stars 342 forks source link

Why do we need to compile the tokens to FST in wsj recipe? #219

Closed Joefi closed 4 years ago

Joefi commented 4 years ago

Hi, I am reading the recipe of wsj. But I am a little confused about the T.fst. Why do we need to compile the tokens to FST and compose with LG.fst. Could we use LG.fst directly?
Thank you.