search

stickeritis / sticker

Succeeded by SyntaxDot: https://github.com/tensordot/syntaxdot
Other
25 stars 2 forks source link

Sentence iterator adapter. #142

Closed twuebi closed 5 years ago

twuebi commented 5 years ago

This PR introduces a trait SentenceIter that provides adapters for conllx::io::Sentences.

This trait makes it easier to introduce other operations such as shuffling via a buffer or local sorting.

In initial experiments buffered shuffling gave better results while converging faster than linear sampling.

The design is modelled after https://github.com/finalfusion/finalfrontier/blob/master/src/deps.rs