Rust port? - Githubissues

varnamproject / libvarnam

Deprecated. See https://github.com/varnamproject/govarnam

http://www.varnamproject.com

Mozilla Public License 2.0

100 stars 21 forks source link

Rust port? #168

Open asdofindia opened 3 years ago

asdofindia commented 3 years ago

Is a Rust port of varnam a bad idea? If someone's porting varnam into a different language, what should they do differently?

joicemjoseph commented 3 years ago

We(Subin, Kailash and me) had a plan to port Varnam to Go. Also had a conversation with Navaneeth on the architecture as well. AFAIK, core of this project is a lexer(I guess) written in Ruby. It leverages advantages of that language. porting the rule engine will be the a bottleneck I guess. It will be great to port to modern languages, either go or rust.

subins2000 commented 3 years ago

If we're going to do a rewrite, some improvements I suggest :

No need of a pattern => word DB (learnings file). Instead, just need a word dictionary.
Add more patterns to VST (Varnam Symbol Table). Prioritized letters n => ന, ണ. Capitalized N will always give ണ. So pani will give suggestions in priority : പനി, പണി. Currently if only the learnings DB has pani assigned to both words will give the different outputs.
When an input say pani is given to varnam, it should tokenize to പനി and പണി using just VST, and then look up the word dictionary to find words starting with പനി and പണി and give additional suggestions.
For english words like "Cricket", the tokenization will give bad results, in such cases we can maybe use a pattern => word DB like the current learnings DB.

subins2000 commented 3 years ago

Also, can dynamic libraries be made in Go & Rust ? Can it then be compiled in Android to be used in Indic Keyboard ? A straight replacement ? Is there a JNI equivalent for Go/Rust libraries in Java ? If not, things will be more difficult.

asdofindia commented 3 years ago

There's https://github.com/mozilla/rust-android-gradle for rust in android

subins2000 commented 3 years ago

Varnam desktop app, varnamd API server is all written in Go. If we're going to do a rewrite, golang will be the better choice for compatibility reasons and maintainability. I saw that there is JNI compatibility with go too.

I hope it will be a straight, simple replacement or others that depend on libvarnam breaks! (Desktop, Indic Keyboard, API) :sweat_smile:

asdofindia commented 3 years ago

@joicemjoseph , you're right about the lexer. https://asd.learnlearn.in/varnam-1/