My question: when training the Predictor, what exactly are source-vocab-size and target-vocab-size supposed to mean: the number of (all) tokens or the number of unique words in the training corpora?
Perhaps adding this to the relevant docs page would be helpful to less experienced users of frameworks of this kind.
Hi,
First of all, thank you for OpenKiwi.
My question: when training the Predictor, what exactly are source-vocab-size and target-vocab-size supposed to mean: the number of (all) tokens or the number of unique words in the training corpora?
Perhaps adding this to the relevant docs page would be helpful to less experienced users of frameworks of this kind.
Best, Andras (TAUS)