Closed SeekPoint closed 3 years ago
what's the meaning of the first question? I think you question is why the word index is sorted by word frequency, am i right? Actually, descend order makes sure common words using litter index, which can reduce memory used in token representation
what's the meaning of the first question? I think you question is why the word index is sorted by word frequency, am i right? Actually, descend order makes sure common words using litter index, which can reduce memory used in token representation