I need text sources - Githubissues

eklem / stopword-sami

Sami stopword lists for natural language processing. Examples on use could be search engines, machine learning and chatbots.

MIT License

1 stars 0 forks source link

I need text sources #17

Open eklem opened 2 years ago

eklem commented 2 years ago

If I get ahold of more text or a site I can crawl for any of the other languages than North-, Lule- and South Sami, I'll create a stopword list for those too.

And doesn't matter if the language is not spoken in Norway.

gunnarvelle commented 2 years ago

Sitemap-listings for southern sami on ndla.no

eklem commented 2 years ago

Thank you @gunnarvelle ! I'll include the content that has longer text and where I can be pretty sure it's only Southern Sami language.

eklem commented 2 years ago

Check Sami newspapers!

eklem commented 2 years ago

Depend on corpus-sma and corpus-smj

For each: copy all files to one folder: cp -r ./source/*.xml ./destination
Open all files and get body, p, string (not object)
Store to JSON-file.
Use in word-count

eklem commented 2 years ago

External library: corpus-smj-sma-json. Will be new dependency.