Closed PhyloStar closed 3 years ago
@xrotwang Thank you. A question on orthography profile. What happens if a doculect does not have a orthography profile? EASTERNBALOCHI does not have one but still the segmentation and IPA transcription seems to work.
Handling Eastern Bengali
: When there are synonyms for a single meaning, then the if statement does not handle it. A proposal is to place the if statement before the if not language.strip():
block so that the language
variable still gets EasternBengali
. Otherwise, synonym for Sun
is ignored.
Regarding orthography profiles: There's a default/fallback profile here https://github.com/lexibank/lsi/blob/master/etc/orthography.tsv - that may just be a remnant of a time with no language-specific profiles, so may not be intended.
Thanks. I will copy the EASTERN BENGALI profile for EASTERNBALOCHI and edit it to get a new profile. Then, we can proceed with experiments.
Can you please look at the creation of reference trees? #18 Is it possible to extract a subtree from glottolog for a list of glottocodes using Pyglottolog? Otherwise, I will get a student to work on it.
Eastern Balochi is here: https://github.com/lexibank/lsi/blob/master/etc/languages.tsv#L274
Eastern Bengali is here: https://github.com/lexibank/lsi/blob/master/etc/languages.tsv#L334
"Eastern" is the language name in LSI but we can identify that it is Balochi or Bengali since through the indentation in the LSI.
Another way to identify Balochi from Bengali is from the "numberInSource". Bengali has
546.
and Balochi has366.
We will need the following changes:
Update the line here: https://github.com/lexibank/lsi/blob/master/lexibank_lsi.py#L83
if number[:4] == "546.": language = "Bengali, Eastern"
Update the entry for
Eastern Bengali
in https://github.com/lexibank/lsi/blob/master/etc/languages.tsv to the following. The first column is updated to avoid conflict withEastern
of Balochi.Bengali, Eastern EASTERN_BENGALI 333 Indo-European Family, Aryan sub-family Indo-Aryan 546. bgp east2744
The orthography file for
EASTERN_BALOCHI
is missing inetc/orthography
directory. @lingulist: How do we go about this? We need to create a orthography file and update the one forEASTERN_BENGALI
since the orthography profile forEASTERN_BENGALI
is made fromEASTERN_BALOCHI
data as well.I think this is the only remaining part. Once we get this done, I will rerun our ICSTLL analyses and add to the paper.