search

facebookresearch / gtn

Automatic differentiation with weighted finite-state transducers.
MIT License
453 stars 40 forks source link

Will GTN be a replacement for OpenFST? #24

Closed zhwa closed 3 years ago

zhwa commented 3 years ago

Looking at the doc, it seems to me that the functions share some similarities with those in OpenFST. Will this lib be an improved version of OpenFST?

Thanks!

awni commented 3 years ago

Indeed, the GTN API is definitely inspired by OpenFST. We didn't explicitly design GTN with the goal of replacing OpenFST. OpenFST still has a lot of functionality that GTN does not and is well optimized for many settings which GTN is not (yet).

In settings where it makes sense to use WFSTs at training time, we hope that GTN can be used for inference as well to minimize the "train-test gap". Ideally one could implement the graph operations once and use them at both training and test time without needing to switch codes. So anywhere that OpenFST is currently used where the WFST could be used to some greater advantage at training time, then GTN may be a better option. However, GTN will still be slower for many situations so this is a goal which we haven't achieved yet.

zhwa commented 3 years ago

Thanks @awni !

Sorry I have another silly question: as OpenFST can be used as part of the WFST decoder for a CTC trained network. For example, the EESEN project implemented speech recognizer using this approach. The WFST decoder itself is trained independently with the RNN. With GTN, is it possible that the full system (RNN + WFST) could be optimized in an end to end paradigm?

awni commented 3 years ago

With GTN, is it possible that the full system (RNN + WFST) could be optimized in an end to end paradigm

Yes exactly! This is one of the main points of this library :).