search

joethei / obsidian-tagcloud

Show a cloud of your tags/words/links in a note
GNU General Public License v3.0
51 stars 3 forks source link

Ability to purge low count keywords to reduce data.json size #12

Open jwhco opened 1 year ago

jwhco commented 1 year ago

The data.json file is getting larger than 100 MB limit on GitHub. Looking into the file I see single word keyword table. Keywords like "He", "a", and "is" don't add value to tag clouds. An option to omit keyword counts under a certain number and/or words under a certain number of characters would reduce file size. This kind of enhancement will lead to a smaller file size plus help user tune tag cloud.

jwhco commented 1 year ago

It would also be useful to have an upper limit on keyword counts. Some keywords with thousands of counts are low value. At a minimum short words tend to have least value, especially transition words and verbs.

jwhco commented 1 year ago

The big problem is maintaining both withStopwords and withoutStopwords in the data.json for every document in a repository. The trim of word length needs to happen before that point. This will reduce the file size. Plus, if there is one instance of something, I'd rather purge that then note it.