Closed nataliacp closed 8 years ago
amaliaskilton
commented
8 years ago I have not updated but will do so today (btwn now and 1pm Pacific time).
On Fri, Feb 12, 2016 at 8:21 AM, Natalia Chousou-Polydouri < notifications@github.com> wrote:
Assigned #8 https://github.com/digling/tukano-project/issues/8 to @amaliaskilton https://github.com/amaliaskilton.
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#event-548735973.
thiagochacon
commented
8 years ago Amalia was supposed to apply tye transformation. I can do it now if she has not done (Ijust realize it afyer seeing the format of the data in the Reflex spreadsheet). Amalia, could you confirm if you have done the transformations? If not, Natalia, could you plaese generate the spreadsheet for kubeo?
Enviado do meu smartphone Samsung Galaxy.
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8
amaliaskilton
commented
8 years ago I am working on this right now.
On Fri, Feb 12, 2016 at 10:14 AM, thiagochacon notifications@github.com wrote:
Amalia was supposed to apply tye transformation. I can do it now if she has not done (Ijust realize it afyer seeing the format of the data in the Reflex spreadsheet). Amalia, could you confirm if you have done the transformations? If not, Natalia, could you plaese generate the spreadsheet for kubeo?
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: Natalia Chousou-Polydouri notifications@github.comData: 12/02/2016 11:21 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comAssunto: [tukano-project] Kubeo quasi-phonemic representations (#8)My latest information regarding Kubeo, is that it is otherwise ready, except that not all items have a FUN form. Could you update if these forms have been generated, so we can proceed in the parsing of the file?
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183436332 .
thiagochacon
commented
8 years ago Thank you
Enviado do meu smartphone Samsung Galaxy.
On Fri, Feb 12, 2016 at 10:14 AM, thiagochacon notifications@github.com wrote:
Amalia was supposed to apply tye transformation. I can do it now if she has not done (Ijust realize it afyer seeing the format of the data in the Reflex spreadsheet). Amalia, could you confirm if you have done the transformations? If not, Natalia, could you plaese generate the spreadsheet for kubeo?
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: Natalia Chousou-Polydouri notifications@github.comData: 12/02/2016 11:21 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comAssunto: [tukano-project] Kubeo quasi-phonemic representations (#8)My latest information regarding Kubeo, is that it is otherwise ready, except that not all items have a FUN form. Could you update if these forms have been generated, so we can proceed in the parsing of the file?
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183436332 .
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183436810
amaliaskilton
commented
8 years ago I have now spent a little over an hour trying to apply Thiago's changes to
the quasi-phonemic representations (having made initial attempts earlier as
well), and it is not a simple process. Thiago, your Kubeo data in the
740-item list appears to have an encoding issue - when I search for <bũ>,
typing the nasality symbol using the IPA Unicode keyboard, I find every
token of
On Fri, Feb 12, 2016 at 10:16 AM, thiagochacon notifications@github.com wrote:
Thank you
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: amaliaskilton notifications@github.comData: 12/02/2016 13:15 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comCc: thiagochacon thiago_chacon@hotmail.comAssunto: Re: [tukano-project] Kubeo quasi-phonemic representations (#8)I am working on this right now.On Fri, Feb 12, 2016 at 10:14 AM, thiagochacon notifications@github.com wrote:
Amalia was supposed to apply tye transformation. I can do it now if she has not done (Ijust realize it afyer seeing the format of the data in the Reflex spreadsheet). Amalia, could you confirm if you have done the transformations? If not, Natalia, could you plaese generate the spreadsheet for kubeo?
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: Natalia Chousou-Polydouri < notifications@github.com>Data: 12/02/2016 11:21 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comAssunto: [tukano-project] Kubeo quasi-phonemic representations (#8)My latest information regarding Kubeo, is that it is otherwise ready, except that not all items have a FUN form. Could you update if these forms have been generated, so we can proceed in the parsing of the file?
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8
— Reply to this email directly or view it on GitHub < https://github.com/digling/tukano-project/issues/8#issuecomment-183436332> .
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183436810
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183437288 .
LinguList
commented
8 years ago This is one of the problems of spreadsheets, since they don't show you what is underlyingly used. There are basically two main types of ũ, one, where it is spelled as u + the nasality, one where there's only one character. I don't know, how spreadsheets handle this, so the only thing that could be done is exporting the sheet to text-form, normalizing to one variant (there are predefined software functions for this), and re-importing...
Try, as an alternative, to just go for one of the obvious ũ variants IN the data, copy paste it, and you should find the instances (at least I'd start from that, assuming that the coding is still consistent)...
thiagochacon
commented
8 years ago Multiple keyboard layouts as we used also different computers!! Let me try to do this, Amalia. And thanks for the effort. Natalia, when can please generate thebkubei spreadsheet!
Enviado do meu smartphone Samsung Galaxy.
On Fri, Feb 12, 2016 at 10:16 AM, thiagochacon notifications@github.com wrote:
Thank you
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: amaliaskilton notifications@github.comData: 12/02/2016 13:15 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comCc: thiagochacon thiago_chacon@hotmail.comAssunto: Re: [tukano-project] Kubeo quasi-phonemic representations (#8)I am working on this right now.On Fri, Feb 12, 2016 at 10:14 AM, thiagochacon notifications@github.com wrote:
Amalia was supposed to apply tye transformation. I can do it now if she has not done (Ijust realize it afyer seeing the format of the data in the Reflex spreadsheet). Amalia, could you confirm if you have done the transformations? If not, Natalia, could you plaese generate the spreadsheet for kubeo?
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: Natalia Chousou-Polydouri < notifications@github.com>Data: 12/02/2016 11:21 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comAssunto: [tukano-project] Kubeo quasi-phonemic representations (#8)My latest information regarding Kubeo, is that it is otherwise ready, except that not all items have a FUN form. Could you update if these forms have been generated, so we can proceed in the parsing of the file?
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8
— Reply to this email directly or view it on GitHub < https://github.com/digling/tukano-project/issues/8#issuecomment-183436332> .
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183436810
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183437288 .
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183458245
thiagochacon
commented
8 years ago That is what I will do. Thanks, Mattis.
Enviado do meu smartphone Samsung Galaxy.
Try, as an alternative, to just go for one of the obvious ũ variants IN the data, copy paste it, and you should find the instances (at least I'd start from that, assuming that the coding is still consistent)...
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183459385
amaliaskilton
commented
8 years ago Thanks Mattis. Yes, the encoding appears to be treating every combination of vowel and diacritics as a distinct, single character. Since there are 2 diacritic features (nasality and tone), each has 2 values, absence is distinctive for both diacritics, and the 2 features combine, this means that each vowel has 6 variants that are all being treated as distinct characters: <u ù ú ũ ũ̀ ṹ>. Can you specify what to do to automatically normalize all the cases?
On Fri, Feb 12, 2016 at 11:33 AM, Johann-Mattis List < notifications@github.com> wrote:
This is one of the problems of spreadsheets, since they don't show you what is underlyingly used. There are basically two main types of ũ, one, where it is spelled as u + the nasality, one where there's only one character. I don't know, how spreadsheets handle this, so the only thing that could be done is exporting the sheet to text-form, normalizing to one variant (there are predefined software functions for this), and re-importing...
Try, as an alternative, to just go for one of the obvious ũ variants IN the data, copy paste it, and you should find the instances (at least I'd start from that, assuming that the coding is still consistent)...
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183459385 .
LinguList
commented
8 years ago You need to do that in a programming language, I don't know whether there is a normalization procedure in google-docs. So it has to be done anyway, but I think Seb will include that upon import, if not, it should be done, but it's no deal to do it from a programming language like python, but one needs to export the sheet and re-import it, and while doing so, one may loose things like comments, or whatever.
amaliaskilton
commented
8 years ago I do not know Python so I'll need to ask one of you (Mattis and Seb) to handle the normalization. After that I can do the phonemic to quasi-phonemic transformations just using grep.
On Fri, Feb 12, 2016 at 11:42 AM, Johann-Mattis List < notifications@github.com> wrote:
You need to do that in a programming language, I don't know whether there is a normalization procedure in google-docs. So it has to be done anyway, but I think Seb will include that upon import, if not, it should be done, but it's no deal to do it from a programming language like python, but one needs to export the sheet and re-import it, and while doing so, one may loose things like comments, or whatever.
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183462231 .
LinguList
commented
8 years ago I can do it, but I need the text-file of the spreadsheet. But alternatively, you could use grep in the same way, just specifying the combined characters, just by taking them from the field inside.
If you want it normalized, please attach the spreadsheet in tab-separated for to teh github-thread, or upload it on github (open a file and paste it there, specifying file-endings as ".tsv", or send it via email.
But if @thiagochacon will have a look at it, it may even not be important at this stage.
nataliacp
commented
8 years ago From what i understand, Seb automatically breaks down those complex characters for reflex. but, I don't know if he has all these combinations. I am confused with the status of Kubeo. Is it ready to generate the spreadsheet or not? I thought we had to wait because not all items had FUN forms. the way i have dealt with this in google spreadsheets is to search and replace to regularize, but it would be a bit crazy with all these combinations. I will ask Seb on Monday for this, unless you find a solution by then.
On Fri, Feb 12, 2016 at 8:42 PM, Johann-Mattis List < notifications@github.com> wrote:
You need to do that in a programming language, I don't know whether there is a normalization procedure in google-docs. So it has to be done anyway, but I think Seb will include that upon import, if not, it should be done, but it's no deal to do it from a programming language like python, but one needs to export the sheet and re-import it, and while doing so, one may loose things like comments, or whatever.
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183462231 .
amaliaskilton
commented
8 years ago @nataliacp, the encoding issue I described above is making it logistically difficult to produce the quasi-phonemic representations. It needs to be regularized first so that I can transform the phonological to quasi-phonemic without manually finding a token of every character.
On Fri, Feb 12, 2016 at 12:02 PM, Natalia Chousou-Polydouri < notifications@github.com> wrote:
From what i understand, Seb automatically breaks down those complex characters for reflex. but, I don't know if he has all these combinations. I am confused with the status of Kubeo. Is it ready to generate the spreadsheet or not? I thought we had to wait because not all items had FUN forms. the way i have dealt with this in google spreadsheets is to search and replace to regularize, but it would be a bit crazy with all these combinations. I will ask Seb on Monday for this, unless you find a solution by then.
On Fri, Feb 12, 2016 at 8:42 PM, Johann-Mattis List < notifications@github.com> wrote:
You need to do that in a programming language, I don't know whether there is a normalization procedure in google-docs. So it has to be done anyway, but I think Seb will include that upon import, if not, it should be done, but it's no deal to do it from a programming language like python, but one needs to export the sheet and re-import it, and while doing so, one may loose things like comments, or whatever.
— Reply to this email directly or view it on GitHub < https://github.com/digling/tukano-project/issues/8#issuecomment-183462231> .
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183468880 .
nataliacp
commented
8 years ago Yes, so I am going to see if we have any ready-made machinery to do the regularization with Seb. then you can make the FUN forms. I hope I will have news for that on Monday.
On Fri, Feb 12, 2016 at 9:24 PM, amaliaskilton notifications@github.com wrote:
@nataliacp, the encoding issue I described above is making it logistically difficult to produce the quasi-phonemic representations. It needs to be regularized first so that I can transform the phonological to quasi-phonemic without manually finding a token of every character.
On Fri, Feb 12, 2016 at 12:02 PM, Natalia Chousou-Polydouri < notifications@github.com> wrote:
From what i understand, Seb automatically breaks down those complex characters for reflex. but, I don't know if he has all these combinations. I am confused with the status of Kubeo. Is it ready to generate the spreadsheet or not? I thought we had to wait because not all items had FUN forms. the way i have dealt with this in google spreadsheets is to search and replace to regularize, but it would be a bit crazy with all these combinations. I will ask Seb on Monday for this, unless you find a solution by then.
On Fri, Feb 12, 2016 at 8:42 PM, Johann-Mattis List < notifications@github.com> wrote:
You need to do that in a programming language, I don't know whether there is a normalization procedure in google-docs. So it has to be done anyway, but I think Seb will include that upon import, if not, it should be done, but it's no deal to do it from a programming language like python, but one needs to export the sheet and re-import it, and while doing so, one may loose things like comments, or whatever.
— Reply to this email directly or view it on GitHub <
https://github.com/digling/tukano-project/issues/8#issuecomment-183462231>
.
— Reply to this email directly or view it on GitHub < https://github.com/digling/tukano-project/issues/8#issuecomment-183468880>
.
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183475769 .
LinguList
commented
8 years ago If not, and if Seb doesn't know of this (what I would doubt), let me know, since if he's using Python for the conversion, it's just one line of code to be applied, and I can just point to it.
BTW: In case you are okay with that, the code for conversion might be put on github directly. In this way, I could have a look and directly help with LingPy-related issues...
thiagochacon
commented
8 years ago Sorry to hear about these issues. Natalia, if you could just generate a spreadsheet of Kubeo, separating each entry in a cell, I could give it a try. Some karapana forms were missing FUN too. My problem with the spreadsheet now is that we have multiple entries in a single cell.
Enviado do meu smartphone Samsung Galaxy.
BTW: In case you are okay with that, the code for conversion might be put on github directly. In this way, I could have a look and directly help with LingPy-related issues...
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183494444
nataliacp
commented
8 years ago ok, we will parse the Kubeo data and try to regularize the symbols to be additive. then you can make all adjustments in the quasi-phonemic forms and then it can go to Mattis for the segmentation.
On Sat, Feb 13, 2016 at 6:13 AM, thiagochacon notifications@github.com wrote:
Sorry to hear about these issues. Natalia, if you could just generate a spreadsheet of Kubeo, separating each entry in a cell, I could give it a try. Some karapana forms were missing FUN too. My problem with the spreadsheet now is that we have multiple entries in a single cell.
Enviado do meu smartphone Samsung Galaxy.
-------- Mensagem original --------De: Johann-Mattis List notifications@github.comData: 12/02/2016 16:29 (GMT-05:00)Para: digling/tukano-project tukano-project@noreply.github.comCc: thiagochacon thiago_chacon@hotmail.comAssunto: Re: [tukano-project] Kubeo quasi-phonemic representations (#8)If not, and if Seb doesn't know of this (what I would doubt), let me know, since if he's using Python for the conversion, it's just one line of code to be applied, and I can just point to it.BTW: In case you are okay with that, the code for conversion might be put on github directly. In this way, I could have a look and directly help with LingPy-related issues...
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-183494444
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-183589980 .
nataliacp
commented
8 years ago Seb parsed the Kubeo data today and transformed all the combined characters in additive. So, I think now you should be able to apply all the rules for the quasi-phonemic forms. once this is done, we can fully check the segmentation with Mattis's tool. The Kubeo file is in the language files directory.
nataliacp
commented
8 years ago some more information for the Kubeo importation template: %% have gone to the CSA field and need sorting between the CSA and the CSO field according to the source @@ have gone to the CSO field
The LAX tags are still in the unified translation, as Seb hasn't modified the script yet. So, you can either copy paste them into the CSA field (if they are not that many) or we will find a solution to do that automatically.
and finally a question, do you want eventually the FUN field to be copied into the PHT (phonetic) field or not?
thiagochacon
commented
8 years ago Thanks for the update. Kubeo quasi-phoeic is pretty close to phonetic but not quite. We will need to apply trqnaformations if we want to have FUN there.
Enviado do meu smartphone Samsung Galaxy.
Reply to this email directly or view it on GitHub: https://github.com/digling/tukano-project/issues/8#issuecomment-184352173
nataliacp
commented
8 years ago My understanding is that the forms currently in FUN are already quasi-phonemic, or they should be. The question is if for this language the quasi-phonemic and the phonetic forms are the same. If yes, we could copy the FUN into the PHT at the end, otherwise we will leave it blank.
amaliaskilton
commented
8 years ago @nataliacp Thanks for fixing the encoding. I will do the quasi-phonemic changes now.
On Tue, Feb 16, 2016 at 5:34 AM, Natalia Chousou-Polydouri < notifications@github.com> wrote:
My understanding is that the forms currently in FUN are already quasi-phonemic, or they should be. The question is if for this language the quasi-phonemic and the phonetic forms are the same. If yes, we could copy the FUN into the PHT at the end, otherwise we will leave it blank.
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-184684519 .
thiagochacon
commented
8 years ago @amaliaskilton I have finished it, Amalia. Sorry if you also did it.
@nataliacp what should be next for Kubeo? the same process as for Karapana?
As for the FUN and pHM, I guess we should keep quais phonemic and phonetic as distinct. so, no, don't copy FUN to PHM
amaliaskilton
commented
8 years ago The Kubeo.tsv file on github still does not have quasi-phonemic representations beyond the first 100 entries or so. @thiagochacon, you made modifications somewhere else?
On Tue, Feb 16, 2016 at 8:02 AM, thiagochacon notifications@github.com wrote:
@amaliaskilton https://github.com/amaliaskilton I have finished it, Amalia. Sorry if you also did it.
@nataliacp https://github.com/nataliacp what should be next for Kubeo? the same process as for Karapana?
As for the FUN and pHM, I guess we should keep quais phonemic and phonetic as distinct. so, no, don't copy FUN to PHM
— Reply to this email directly or view it on GitHub https://github.com/digling/tukano-project/issues/8#issuecomment-184745633 .
thiagochacon
commented
8 years ago I am doing the modifications and the template adjustments right now in my desktop : )
Date: Tue, 16 Feb 2016 08:05:42 -0800 From: notifications@github.com To: tukano-project@noreply.github.com CC: thiago_chacon@hotmail.com Subject: Re: [tukano-project] Kubeo quasi-phonemic representations (#8)
The Kubeo.tsv file on github still does not have quasi-phonemic
representations beyond the first 100 entries or so. @thiagochacon, you made
modifications somewhere else?
On Tue, Feb 16, 2016 at 8:02 AM, thiagochacon notifications@github.com
wrote:
@amaliaskilton https://github.com/amaliaskilton I have finished it,
Amalia. Sorry if you also did it.
@nataliacp https://github.com/nataliacp what should be next for Kubeo?
the same process as for Karapana?
As for the FUN and pHM, I guess we should keep quais phonemic and phonetic
as distinct. so, no, don't copy FUN to PHM
—
Reply to this email directly or view it on GitHub
https://github.com/digling/tukano-project/issues/8#issuecomment-184745633
.
— Reply to this email directly or view it on GitHub.
nataliacp
commented
8 years ago yes, the next step is to adjust the FUN forms and any other columns that need sorting, making sources uniform, lax rows, polysemous words and the like.
thiagochacon
commented
8 years ago I have adjusted Kubeo data according to Reflex templates and guidelines. I will open a new issue now and close this one.
My latest information regarding Kubeo, is that it is otherwise ready, except that not all items have a FUN form. Could you update if these forms have been generated, so we can proceed in the parsing of the file?