Missing mappings for stemmed languages

[sigdestad]

During documentation, the following languages seems to be missing from the code:

• Greek - el
• Indonesian - id
• Italian - it
• Kurdish aka Sorani - ku

Add to standard repo mappings before release of XP7

[GlennRicaud]

So after review here are the remarks:

• 7 Missing languages: greek, indonesian, italian, sorani (subset of kurdisk), chinese, japanese, korean
• The 2 variations of norwegian (Bokmål and nynorsk) are not handled
• Brazilian is not used. The words are treated as portuguese
• Custom analyzer implemented based on a 6.7 reference (https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-lang-analyzer.html)
• No need to define custom analyzers/stemmers/... There are equivalent built-in analyzers

Actions:

• Remove custom analyzer and replace by built-in analyzers
• Add missing mappings (See mapping list below)
• Add custom stemmer/analyzer for nynorsk
• Refactor the Controller/Parser code to handle these changes

Final ISO 639-1 code to analyzer mapping: https://www.elastic.co/guide/en/elasticsearch/reference/2.4/analysis-lang-analyzer.html

ar -> arabic
hy -> armenian
eu -> basque
bn -> bengali
pt-BR ->  brazilian   // This is not working. Brazilian words will be put as portuguese "pt". Left it in the mappings and settings in case we decide to refactor in the future.
bg -> bulgarian
ca -> catalan
zh -> cjk //3 missing CJK languages
ja -> cjk
ko -> cjk
cs -> czech
da -> danish
nl -> dutch
en -> english
fi -> finnish
fr -> french
gl -> galician
de -> german
el -> greek
hi -> hindi
hu -> hungarian
id -> indonesian
ga -> irish
it -> italian
lv -> latvian
lt -> lithuanian
no -> norwegian
  nb -> norwegian
  nn -> //Custom analyzer based on language "light_nynorsk"
fa -> persian
pt -> portuguese
ro -> romanian
ru -> russian
ku -> sorani   //Sorani is not exactly kurdisk. More of a subset
es -> spanish
sv -> swedish
tr -> turkish
th -> thai