Attempting truncate_input_tokens=2 (or 1) was creating a strange error (or misbehaving) because it takes at least 3 tokens for [CLS] TOK [SEP] for meaningful results.
Now that truncate value generally means number of tokens not including begin/end.
On the max end the 2 special tokens will be allowed to consume 2 from the limit.
Batch embedding processing was returning odd/misordered results when combined with truncation. Added a re tokenize() call to avoid sending the overflow tokens as features to be processed.