search

garvys-org / rustfst

Rust re-implementation of OpenFST - library for constructing, combining, optimizing, and searching weighted finite-state transducers (FSTs). A Python binding is also available.
https://docs.rs/rustfst
Other
141 stars 16 forks source link

Support LogWeight in C and Python #233

Open dhdaines opened 1 year ago

dhdaines commented 1 year ago

Writing code in Rust is fun, but writing it in Python is more fun... but for many use cases (i.e. training or constructing WFSTs for language modeling) we need the log semiring which is only accessible from Rust.

Evidently to support it elsewhere requires it to be added to rustfst-ffi which implies duplicating all the classes and functions everywhere. Since log and tropical semiring are nearly the same thing this seems a bit wasteful to me!

As an aside, I wonder whether the OpenFST approach of templating everything based on the weight type is really a good idea, given that it duplicates all the object code simply to have a different implementation of a handful of simple functions, but this is typical of the C++ mentality underlying OpenFST...

Garvys commented 1 year ago

Hello ! Very nice feature request. It would indeed be very useful to have all semiring available in python and not only the tropical one. Regarding implementation, I will have to dig but it seems that we won't have much choice of creating a type for each in the same spirit as what has been done for ConstFst and VectorFst.

dhdaines commented 1 year ago

I may have time to make a PR for this one, it is more immediately useful than FAR support for my purposes (since my collections of FSTs will fit just fine in memory)