rth / vtext

Simple NLP in Rust with Python bindings
Apache License 2.0
146 stars 11 forks source link

Implement IDF transforms #4

Open rth opened 5 years ago

rth commented 5 years ago

It would be necessary to implement IDF transforms, and possibly expose a TfidfVectorizer estimator.

This requires selecting a sparse array library. For now, we use custom CSRArray structs to represent CSR arrays. https://github.com/vbarrielle/sprs is a good candidate but this needs more investigation in any case.