jncraton / languagemodels

Explore large language models in 512MB of RAM
https://jncraton.github.io/languagemodels/
MIT License
1.18k stars 78 forks source link

New Features: lm.get_URL(), lm_get_MD(), lm_get_PDF, lm_getTXT, etc... #33

Open Gustavo-Livemix opened 7 months ago

Gustavo-Livemix commented 7 months ago

It will be wonderfull if there were Helper functions for external retrieval like:

jncraton commented 7 months ago

Thanks for the kind words! I'm glad that you are finding this package useful.

I can certainly see the value of simplifying retrieval from different types of documents. I'm not planning to implement this right now, but I'll consider ways to better integrate this in the future.

As you are probably aware, there are already packages such as textract that specialize in solving the problem of going from all sorts of documents to plain text. Using something like that would hopefully address your immediate needs.