Language names of Languages supported by Fasttext

[AetherPrior]

I am trying to find out the names of languages supported by Fasttext's LID tool, given these language codes listed here:

af als am an ar arz as ast av az azb ba bar bcl be bg bh bn bo bpy br bs bxr ca cbk ce ceb ckb co cs cv cy da de diq dsb dty dv el eml en eo es et eu fa fi fr frr fy ga gd gl gn gom gu gv he hi hif hr hsb ht hu hy ia id ie ilo io is it ja jbo jv ka kk km kn ko krc ku kv kw ky la lb lez li lmo lo lrc lt lv mai mg mhr min mk ml mn mr mrj ms mt mwl my myv mzn nah nap nds ne new nl nn no oc or os pa pam pfl pl pms pnb ps pt qu rm ro ru rue sa sah sc scn sco sd sh si sk sl so sq sr su sv sw ta te tg th tk tl tr tt tyv ug uk ur uz vec vep vi vls vo wa war wuu xal xmf yi yo yue zh

I tried to map the ISO codes to each language, but it seems non-standard, either using ISO-639-1 or ISO-639-3. Does anyone have a list of language names for these codes, or know how to find them?
Wikipedia's list does not cover all of them either, so manual mapping too did not help.
Thanks!

[furas]

some of names you can find in your link Wikipedia's list. Some may find in sublink ISO 639 macrolanguage

But using Google to search code bpy I found page https://iso639-3.sil.org/code_tables/639/data and it may have all names (but I didn't test all of codes from your list)

At the bottom of this page I found also link to download tables and it seems it has tables as iso-639-3.tab (tab seperated values similar to .csv)

[CarlosGDCJ]

You could use app-subtags. Go to this site, paste the entire list in the Look up field and it returns the identified languages. Out of all the tokens, only eml is not identified. Maybe fasttext uses a slightly different version of BCP47 instead of ISO-639.

Alternatively, if you wanna do something similar in python, you could use langcodes:

# pip install langcodes language_data
from langcodes import tag_is_valid, Language

lang_tag = "af"

if tag_is_valid(lang_tag):
  lang_name = Language.get(lang_tag).display_name("en")
  print(f"Language name in english: {lang_name}") # Language name in english: Afrikaans

[kargaranamir]

It's based on the List_of_Wikipedias: https://en.wikipedia.org/wiki/List_of_Wikipedias

for example the als code in the fasttext refers to gsw code in the ISO-639. or ku doesnot refers to kur but refers to kmr

[yunsii]

It's based on the List_of_Wikipedias: en.wikipedia.org/wiki/List_of_Wikipedias

for example the als code in the fasttext refers to gsw code in the ISO-639. or ku doesnot refers to kur but refers to kmr

Great comment! I have created a function to normalize the output identify language (aka WP code) to ISO 639-3 Id, and the languages json data generated by a set of languages scripts.