setup for red-lists - Githubissues

eklem / stopword-sami

Sami stopword lists for natural language processing. Examples on use could be search engines, machine learning and chatbots.

MIT License

1 stars 0 forks source link

setup for red-lists #18

Closed eklem closed 2 years ago

eklem commented 2 years ago

[x] separate files for each red-list. Part of the datasets folder
[x] read + feed it to the swt.getStopwords function
[x] Slice the list on i.e. 500 words in the beginning. Then you'll know if it is good to push further with more redlisted words to "get to" stopwords later in the list/array.

The final list should probably/hopefully contain 200 - 300 words.

eklem commented 2 years ago

Set up an array.filter https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array/filter

eklem commented 2 years ago

Think slicing shoul be part of swt.getStopwords function. So from swt.getStopwords(wordsCounted.words, redlist) to swt.getStopwords(wordsCounted.words, redlist, maxWords) where maxWords is an integer.