keymanapp / keyboards

Open Source Keyman keyboards
152 stars 276 forks source link

bug: keyboard search includes sil_euro_latin when searching for "han script" #3221

Open darcywong00 opened 2 hours ago

darcywong00 commented 2 hours ago

I was testing localizing keyboard searches and didn't understand why sil_euro_latin appeared in the list when searching for s:han (han script)

Repros on the live site: https://keyman.com/keyboards?q=s%3Ahan

Image

I'm not finding "han script" in the api.keyman.com blob https://api.keyman.com/keyboard/sil_euro_latin

mcdurdin commented 2 hours ago

This happens because sil_euro_latin includes zhd.

See the api search result -- the match object for sil_euro_latin:

 "match": {
        "name": "Han",
        "type": "script",
        "weight": 10,
        "downloads": 1041,
        "totalDownloads": 47470,
        "finalWeight": 79.4889722231331,
        "tag": "zhd"
      }

And looking at sil_euro_latin.kps:

        <Language ID="zhd">Dai Zhuang</Language>

So, this is a bug in sil_euro_latin -- because langtags.json shows that the default script for zhd is in fact Hani (Han script):

    {
        "full": "zhd-Hani-CN",
        "iana": [ "Dai Zhuang" ],
        "iso639_3": "zhd",
        "macrolang": "za",
        "name": "Zhuang, Dai",
        "names": [ "Bu Dai", "Dai Zhuang", "Kau Ndae", "Khaau Daai", "Thu Lao", "Tu", "Tuliao", "Tuzu", "Wen-Ma Southern Zhuang", "Zhuangyu Nanbu Fangyan Wen-Ma Tuyu", "Zhuangyu Nanbu fangyan Dejing tuyu" ],
        "region": "CN",
        "regionname": "China",
        "regions": [ "VN" ],
        "script": "Hani",
        "sldr": false,
        "tag": "zhd",
        "tags": [ "zhd-CN", "zhd-Hani" ],
        "windows": "zhd-Hani"
    },
    {
        "full": "zhd-Latn-VN",
        "iana": [ "Dai Zhuang" ],
        "iso639_3": "zhd",
        "macrolang": "za",
        "name": "Zhuang, Dai",
        "names": [ "Bu Dai", "Dai Zhuang", "Kau Ndae", "Khaau Daai", "Thu Lao", "Tu", "Tuliao", "Tuzu", "Wen-Ma Southern Zhuang", "Zhuangyu Nanbu Fangyan Wen-Ma Tuyu", "Zhuangyu Nanbu fangyan Dejing tuyu" ],
        "obsolete": true,
        "region": "VN",
        "regionname": "Viet Nam",
        "regions": [ "CN" ],
        "script": "Latn",
        "sldr": false,
        "tag": "zhd-Latn",
        "tags": [ "zhd-VN" ],
        "windows": "zhd-Latn"
    },
mcdurdin commented 2 hours ago

Note: I haven't verified the other tags. It'd be good for a package build to verify the script tags are the same for all referenced BCP 47 languages (and hint on mismatches)

mcdurdin commented 35 minutes ago

After implementing a test in kmc-keyboard-info against keymanapp/keyman#12752, I got the following report:

$ ./build.sh | grep KM09011
aramaic_hebrew.kpj - hint KM09011: The script 'Syrj' associated with language tag 'amw-Syrj' does not match the script 'Syrc' for the first language in the package.
arbore.kpj - hint KM09011: The script 'Latn' associated with language tag 'amf' does not match the script 'Zyyy' for the first language in the package.
batak.kpj - hint KM09011: The script '<param>' associated with language tag 'btk-Batk' does not match the script 'Batk' for the first language in the package.
baybayin.kpj - hint KM09011: The script 'Latn' associated with language tag 'tbw-Tglg' does not match the script 'Tglg' for the first language in the package.
baybayin.kpj - hint KM09011: The script 'Latn' associated with language tag 'hnn-Tglg' does not match the script 'Tglg' for the first language in the package.
baybayin.kpj - hint KM09011: The script 'Latn' associated with language tag 'bku-Tglg' does not match the script 'Tglg' for the first language in the package.
bu_phonetic.kpj - hint KM09011: The script '<param>' associated with language tag 'und-Latn' does not match the script 'Latn' for the first language in the package.
basic_kbdinuk2.kpj - hint KM09011: The script 'Cans' associated with language tag 'iu' does not match the script 'Latn' for the first language in the package.
basic_kbdiulat.kpj - hint KM09011: The script 'Cans' associated with language tag 'iu' does not match the script 'Latn' for the first language in the package.
basic_kbdlisub.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'lbc-Lisu' does not match the script 'Lisu' for the first language in the package.
basic_kbdlisus.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'lbc-Lisu' does not match the script 'Lisu' for the first language in the package.
basic_kbdtifi2.kpj - hint KM09011: The script 'Latn' associated with language tag 'shi-Latn' does not match the script 'Tfng' for the first language in the package.
deseret.kpj - hint KM09011: The script 'Latn' associated with language tag 'hop-Dsrt' does not match the script 'Dsrt' for the first language in the package.
gandhari.kpj - hint KM09011: The script 'Khar' associated with language tag 'pgd-Latn' does not match the script 'Deva' for the first language in the package.
geezbrhan.kpj - hint KM09011: The script 'Latn' associated with language tag 'suq' does not match the script 'Ethi' for the first language in the package.
geezbrhan.kpj - hint KM09011: The script 'Latn' associated with language tag 'guk' does not match the script 'Ethi' for the first language in the package.
geezbrhan.kpj - hint KM09011: The script 'Latn' associated with language tag 'dwr' does not match the script 'Ethi' for the first language in the package.
gff_ethiopic.kpj - hint KM09011: The script 'Latn' associated with language tag 'amf' does not match the script 'Ethi' for the first language in the package.
gff_ethiopic.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc' does not match the script 'Ethi' for the first language in the package.
gff_ethiopic.kpj - hint KM09011: The script 'Latn' associated with language tag 'sid' does not match the script 'Ethi' for the first language in the package.
gff_ethiopic.kpj - hint KM09011: The script 'Latn' associated with language tag 'jnj' does not match the script 'Ethi' for the first language in the package.
gff_harege_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'amf' does not match the script 'Ethi' for the first language in the package.
gff_harege_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc' does not match the script 'Ethi' for the first language in the package.
gff_harege_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'sid' does not match the script 'Ethi' for the first language in the package.
gff_harege_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'jnj' does not match the script 'Ethi' for the first language in the package.
gff_mesobe_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'amf' does not match the script 'Ethi' for the first language in the package.
gff_mesobe_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc' does not match the script 'Ethi' for the first language in the package.
gff_mesobe_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'sid' does not match the script 'Ethi' for the first language in the package.
gff_mesobe_fidelat.kpj - hint KM09011: The script 'Latn' associated with language tag 'jnj' does not match the script 'Ethi' for the first language in the package.
idc_deseret.kpj - hint KM09011: The script 'Latn' associated with language tag 'hop-Dsrt' does not match the script 'Dsrt' for the first language in the package.
indonesian_suku.kpj - hint KM09011: The script 'Batk' associated with language tag 'btd-Latn' does not match the script 'Latn' for the first language in the package.
indonesian_suku.kpj - hint KM09011: The script 'Arab' associated with language tag 'mfa-Latn' does not match the script 'Latn' for the first language in the package.        
itrans_odia.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxv' does not match the script 'Orya' for the first language in the package.
itrans_roman.kpj - hint KM09011: The script 'Latn' associated with language tag 'hi-Latn' does not match the script 'Deva' for the first language in the package.
itrans_roman.kpj - hint KM09011: The script 'Taml' associated with language tag 'ta-Latn' does not match the script 'Deva' for the first language in the package.
jawa.kpj - hint KM09011: The script 'Java' associated with language tag 'jv-Java' does not match the script 'Latn' for the first language in the package.
jawa.kpj - hint KM09011: The script 'Java' associated with language tag 'kaw-Java' does not match the script 'Latn' for the first language in the package.
jawa.kpj - hint KM09011: The script 'Java' associated with language tag 'su-Java' does not match the script 'Latn' for the first language in the package.
jawa.kpj - hint KM09011: The script 'Java' associated with language tag 'osi-Java' does not match the script 'Latn' for the first language in the package.
jawa.kpj - hint KM09011: The script 'Java' associated with language tag 'tes-Java' does not match the script 'Latn' for the first language in the package.
kawi_inscript.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'omy-Kawi' does not match the script 'Kawi' for the first language in the package.
kawi_inscript.kpj - hint KM09011: The script 'Sund' associated with language tag 'osn-Kawi' does not match the script 'Kawi' for the first language in the package.
kreative_superlatin.kpj - hint KM09011: The script 'Cyrl' associated with language tag 'sr' does not match the script 'Latn' for the first language in the package.
kreative_supersymbol.kpj - hint KM09011: The script 'Zsym' associated with language tag 'zxx-Zsym' does not match the script 'Zmth' for the first language in the package.
mahajani_inscript.kpj - hint KM09011: The script 'Arab' associated with language tag 'lah-Mahj' does not match the script 'Mahj' for the first language in the package.
nailangs.kpj - hint KM09011: The script 'Arab' associated with language tag 'kby' does not match the script 'Latn' for the first language in the package.
soyombo.kpj - hint KM09011: The script 'Deva' associated with language tag 'sa-Soyo' does not match the script 'Soyo' for the first language in the package.
soyombo.kpj - hint KM09011: The script 'Tibt' associated with language tag 'bo-Soyo' does not match the script 'Soyo' for the first language in the package.
syriac_arabic.kpj - hint KM09011: The script 'Syrj' associated with language tag 'amw-Syrj' does not match the script 'Syrc' for the first language in the package.
syriac_phonetic.kpj - hint KM09011: The script 'Syrj' associated with language tag 'amw-Syrj' does not match the script 'Syrc' for the first language in the package.
sil_cameroon_azerty.kpj - hint KM09011: The script 'Bamu' associated with language tag 'bax' does not match the script 'Latn' for the first language in the package.
sil_cameroon_azerty.kpj - hint KM09011: The script 'Arab' associated with language tag 'mfi' does not match the script 'Latn' for the first language in the package.
sil_cameroon_qwerty.kpj - hint KM09011: The script 'Arab' associated with language tag 'mfi' does not match the script 'Latn' for the first language in the package.
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'amf-Latn' does not match the script 'Zyyy' for the first language in the package.
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'anu-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'bcq-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'bst-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'bwo-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'gmv-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'gof-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'gru-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'guk-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'gwd-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'hdy-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'kbr-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'kmq-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'kqy-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'ktb-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'lgn-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'mdx-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'mdy-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'moy-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'mpe-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'muz-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'myf-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'mym-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'nnj-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'nus' does not match the script 'Zyyy' for the first language in the package.      
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'she-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'sid' does not match the script 'Zyyy' for the first language in the package.      
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'so' does not match the script 'Zyyy' for the first language in the package.       
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'ssy' does not match the script 'Zyyy' for the first language in the package.      
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'suq-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'tsb-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'wal-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'wti-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'xom-Latn' does not match the script 'Zyyy' for the first language in the package. 
sil_el_ethiopian_latin.kpj - hint KM09011: The script 'Latn' associated with language tag 'zay-Latn' does not match the script 'Zyyy' for the first language in the package.
sil_ethiopic.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc' does not match the script 'Ethi' for the first language in the package.
sil_ethiopic.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'mul-Ethi' does not match the script 'Ethi' for the first language in the package.
sil_ethiopic_power_g.kpj - hint KM09011: The script 'Latn' associated with language tag 'kxc' does not match the script 'Ethi' for the first language in the package.
sil_ethiopic_power_g.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'mul-Ethi' does not match the script 'Ethi' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Arab' associated with language tag 'fub' does not match the script 'Latn' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Hebr' associated with language tag 'lad' does not match the script 'Latn' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Cyrl' associated with language tag 'sr' does not match the script 'Latn' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Runr' associated with language tag 'sxu' does not match the script 'Latn' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Hani' associated with language tag 'zhd' does not match the script 'Latn' for the first language in the package.
sil_euro_latin.kpj - hint KM09011: The script 'Hani' associated with language tag 'zlj' does not match the script 'Latn' for the first language in the package.
sil_hebr_grek_trans.kpj - hint KM09011: The script 'Cprt' associated with language tag 'grc-Latn' does not match the script 'Grek' for the first language in the package.
sil_hebr_grek_trans.kpj - hint KM09011: The script 'Hebr' associated with language tag 'he-Latn' does not match the script 'Grek' for the first language in the package.     
sil_hebr_grek_trans.kpj - hint KM09011: The script 'Hebr' associated with language tag 'hbo-Latn' does not match the script 'Grek' for the first language in the package.    
sil_indic_roman.kpj - hint KM09011: The script 'Beng' associated with language tag 'as-Latn' does not match the script 'Latn' for the first language in the package.
sil_indic_roman.kpj - hint KM09011: The script 'Beng' associated with language tag 'bn-Latn' does not match the script 'Latn' for the first language in the package.
sil_lisu_basic.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'lbc' does not match the script 'Lisu' for the first language in the package.
sil_lisu_standard.kpj - hint KM09011: The script 'Zyyy' associated with language tag 'lbc' does not match the script 'Lisu' for the first language in the package.
sil_nigeria_odd_vowels.kpj - hint KM09011: The script 'Arab' associated with language tag 'kby' does not match the script 'Latn' for the first language in the package.
sil_pan_africa_mnemonic.kpj - hint KM09011: The script 'Arab' associated with language tag 'fub' does not match the script 'Latn' for the first language in the package.
sil_pan_africa_positional.kpj - hint KM09011: The script 'Arab' associated with language tag 'fub' does not match the script 'Latn' for the first language in the package.

This is an indication that there may be an error in the language tag, but not necessarily conclusive, which is why it is reported only as a HINT.