Guide users to new vocabulary

[StephanAkkerman]

Idea: guide the users in learning new words for their selected language. They either have the option to input words themselves or be guided by this list (or another if there's something better)

Tasks

• Find lists that have many frequent words list (preferably corresponding with our supported languages)
  • https://github.com/frekwencja/most-common-words-multilingual
  • https://github.com/hermitdave/FrequencyWords (any issues here are also found in the original source)
  • https://opus.nlpl.eu/OpenSubtitles/corpus/version/OpenSubtitles (this is where FrequencyWords is based on, 62 langs)
  • Could download it from https://object.pouta.csc.fi/OPUS-OpenSubtitles/v2018/freq/en.freq.gz
  • https://huggingface.co/datasets/Helsinki-NLP/open_subtitles
  • https://en.wiktionary.org/wiki/Wiktionary:Frequency_lists/

Plan:

• Scrape https://github.com/hermitdave/FrequencyWords/tree/master/content/2018
• Make it into huggingface dataset: https://huggingface.co/datasets/StephanAkkerman/frequency-words-2018
• Write down all supported languages
• Check https://en.wiktionary.org/wiki/Wiktionary:Frequency_lists for datasets for missing languages

Workflow:

• Users selects language they want to learn
• If they want to use this frequency word list, then download it from the source (or compile a dataset ourselves)
• Follow the order of the words in that repo (most frequent to least frequent)
• Translate foreign word to English
• Generate mnemonic

Option (instead of translating):

• Find word in scraped Wiktionary data (https://kaikki.org/dictionary/rawdata.html / https://github.com/tatuylonen/wiktextract / https://github.com/abuccts/wikt2pron/tree/master)
• Match the IPA and word meaning
• Use this for the #18

https://github.com/hermitdave/FrequencyWords/blob/master/content/2018/id/id_50k.txt

Supports many languages and probably better for real life scenarios

[StephanAkkerman]

Look into: https://foosoft.net/projects/anki-connect/ to find a method for checking the existing cards in a deck. Could use a combination of:

• findCards
• cardsInfo

An alternative would be to let the user select the words from the list itself (how would we do this?)

[StephanAkkerman]

Maybe pre-process the data with a dictionary, dropping wrong entries, which we can also use for other forms of the same words for the grammar cards (#83)

This should be a separate issue for later

[StephanAkkerman]

Solution: just locally save the card that was added the last, no need to Anki-connect calls that way