Handling Right to Left languages

nlmatics / llmsherpa

Developer APIs to Accelerate LLM Projects

https://www.nlmatics.com

MIT License

1.37k stars 134 forks source link

Handling Right to Left languages #75

Open wolfassi123 opened 5 months ago

wolfassi123 commented 5 months ago

Have the library been tested on Right to Left languages (Arabic, etc.). The parsing seems a bit off when it comes to the arabic language with the letters all over the place (sometimes it's not just reversed).

Anyone faced this issue and used a fix?

EssamWisam commented 3 months ago

I'm facing the same issue with Arabic documents. If you can point us to the issue can try to help with a PR.

@moshewe