[BUG] Default trascription variants and interlinear gloss

stannam commented 2 years ago

Typically, a corpus with variant forms involves three attributes: orthography, default transcription, and alternative transcription. Alternative transcription varies within lexical items.

We also allow variant forms without the alternative transcription attribute. I.e., the user can select canonical transcriptions to vary within lexical items (see #703). The problem in this case is that analysis functions cannot recognize transcription.

Two options

1. No canonical but only variants

When Transcription (default) varies within lexical items, and Transcription (alternative) is not provided,
- In all analysis functions, the 'pronunciation variants' option should allow the variant options, but not 'use canonical forms.'
- Issues
- On the main corpus table, which transcription should be presented? (currently, whatever appears first is shown)
- 'List pronunciation variants' not working.
- How should 'Edit word' work? Which variant should this edit?

2. Do not allow default transcription to vary

Then a corpus must have a non-varying canonical column..

stannam commented 2 years ago

update:

1. Pronunciation variants is only allowed for 'transcription (alternative)' (solved)

We chose path 2. In other words, transcription (default) does NOT vary.
- Following this, commit ff5a7c06e2f7a478c5a239de7a78ef7eb5f4a45e blocks the user from selecting '...vary within lexical items' for anything other than transcription (alternative).
Now, a two-lined interlinear corpus cannot handle pronunciation variants. If an ilg file has two lines, there should be one line for spelling and another for canonical transcription. If the transcription differs, they should be treated as if homographs.

e.g., in the following mini example, two 'tomato's should be different entries <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

Whereas in the example below, one 'tomato' has pronunciation variants (with 'canonical' being [təmeɪɾoʊ]). <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

Also, see 'example_files/variants/interlinear_variants' (in the shared dropbox storage).

2. ILG has no inventory categorization (solved)

Previously, the autocategorize function did not work when the user imported an ilg. All segments went to 'uncategorized segments'
This was an unintended result of my previous commit 129c395c6a32c2bda9d925d7c76054170cc245f6 .
- That was a fix to issues #703 (allow the default transcription vary) and #720
- Before that commit, the autocategorization worked on ilg, but the problem was that any unrecognized symbol would crash PCT.
So I partly reverted that commit so that an ilg corpus can have the inventory chart, and at the same time, unrecognizable symbols do not crash PCT.
- I confirmed in different scenarios that PCT does not crash and all recognizable segments are properly categorized. <html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:x="urn:schemas-microsoft-com:office:excel" xmlns="http://www.w3.org/TR/REC-html40">

	all recognized	partially recognized	none recognized
ilg	✓	✓	✓
csv	✓	✓	✓

kchall commented 2 years ago

Can confirm that all three cases in the Excel file are working for me.

kchall commented 2 years ago

I think this is all working, and the documentation is up to date.

PhonologicalCorpusTools / CorpusTools

[BUG] Default trascription variants and interlinear gloss #787