Open jwhco opened 1 year ago
It would also be useful to have an upper limit on keyword counts. Some keywords with thousands of counts are low value. At a minimum short words tend to have least value, especially transition words and verbs.
The big problem is maintaining both withStopwords
and withoutStopwords
in the data.json
for every document in a repository. The trim of word length needs to happen before that point. This will reduce the file size. Plus, if there is one instance of something, I'd rather purge that then note it.
The
data.json
file is getting larger than 100 MB limit on GitHub. Looking into the file I see single word keyword table. Keywords like "He", "a", and "is" don't add value to tag clouds. An option to omit keyword counts under a certain number and/or words under a certain number of characters would reduce file size. This kind of enhancement will lead to a smaller file size plus help user tune tag cloud.