nschum / auto-dictionary-mode

Emacs: automatic dictionary switcher for flyspell
http://nschum.de/src/emacs/auto-dictionary/
28 stars 13 forks source link

add nn/nb words, from Språkbanken freq-lists; remove some overlaps #19

Open unhammer opened 7 years ago

unhammer commented 7 years ago

New words filtered by Danish/Swedish/other-Norwegian, e.g.

sudo apt install apertium-{dan,swe,nno,nob} # Using http://apertium.projectjj.com/apt/nightly

only_unknown () {
    grep '/\*'
}
only_wordform () {
    sed 's,/\*.*,,; s,/[0-9]*<.*\$,,; s/[][$^]//g'
}

wget http://www.nb.no/sbfil/tekst/1gram_nno_f1_freq.zip
unzip 1gram_nno_f1_freq.zip

<1gram_nno_f1_f.frk iconv -f iso-8859-1 -t utf-8 \
  | apertium-destxt \
  | lt-proc /usr/share/apertium/apertium-dan/dan.automorf.bin \
  | only_unknown \
  | only_wordform \
  | apertium-destxt \
  | lt-proc /usr/share/apertium/apertium-nob/nob.automorf.bin \
  | only_unknown \
  | only_wordform \
  | apertium-destxt \
  | lt-proc /usr/share/apertium/apertium-swe/swe.automorf.bin \
  | only_unknown \
  | only_wordform \
  | head -100  \
  | sed 's/ *[0-9]*  *//' \
  | sed 's/.*/"&"/'