output files (general question)

[thealk]

Maggie and I calibrated using the .csv files today for the first time (her old version up until this week wouldn't allow her to append to the log files, so up to now we've been calibrating by going over vowels together like you and I did Morgan).

After looking over both our files, I'm a little unclear about what the output should be. 1) nothing gets output to corrected/. What files are supposed to go here? 2) in the output CSV files in log/, there seem to be several tokens that don't have the vowel we were looking at. I'm not sure how/why these wind up on the output file (especially since they have a value in the final column of "OK" or "removed" and I never checked them). 3) multisyllabic words show up once for each syllable, even for syllables containing vowels we didn't change. I'm not sure if this is a separate issue than the one above or if they're related. 4) the maxForms doesn't change, even if the number of formants was changed during the editing session and F1 and F2 values have changed as a result. This might be fine (as we don't actually need to know how the formants were measured as long as we can assume they were correctly measured if a change was made), I just want to make sure that's what it's supposed to do.

Does this all sound right? Maggie and I also didn't have all the same tokens in our output files. In some cases I had tokens that she did not (this might be okay because she hadn't finished the speaker, but:) she also had tokens in her files that didn't show up in mine, and we both had tokens that didn't show up in the plot when we reopened that speaker in Plotmish. I'd like to have a better idea of how these are being generated - could you give me a brief breakdown of what you guys expect to be in output files and how you want those values to be used so I can be sure that they're being correctly generated?

[msonderegger]

Huh -- some of these are intentional, but others sound like they could be bugs. I'm on a pretty tight schedule at this conference, but will try to write more about how these are going to be used later today. @mishaschwartz @thealk : could you prioritize communicating about this set of issues and clearing up any bugs? It's of chief importance that any annotation Thea and Maggie are doing be usable. (Misha: maybe it would be easier for annotators to use the CSV file generated by running update_formants.py, so as to not have to worry about all the rows which might be present for a given token?) I will keep an eye on this thread regularly, so please post any questions...

[mishaschwartz]

Hey, I'm a little uncertain exactly what the issue is with some of these but I'll try to clear these up as much as possible: 1)the corrected folder is the output of updateFormants.py. If you're just running plotmish there will be nothing output here but once you've made all the changes you want you can run updateFormants.py to change the original formant.txt files which will then end up in corrected/ 2)This is an error on my part. When you cleared a range of tokens it was also clearing any hidden tokens that also fell in that range. This should now be resolved.
3)This isn't happening for me when I do it. It's possible that this is related to problem 2) if tokens are being removed unintentionally. Please let me know if you keep on having this problem. 4)maxForms will change if you change it using remeasureF. I haven't figured out a way to get it to give this info if you're using remeasureP. Thea, you have more praat scripting experience than me, if you have any ideas as to how we can get praat to somehow output the formant setting info, please let me know.

[thealk]

going through some more detailed test cases. will post an update momentarily.

[thealk]

Okay: Mostly things seem to be looking good, with the exception of measurement information not getting properly logged when the main remeasurement is done by changing the number of formants. Here's a quick breakdown:

1. 2 and 3 above are fixed, thanks. Only the words containing the vowels of interest appear in the output csv, and if they are multisyllabic only the syllable containing the vowel of interest appears.
2. There seems to be an issue logging measurements when remeasuring via changing the max number of formants, regardless of whether they were remeasured in remeasureF or remeasureP:
   • Generally when a remeasurement is made, two rows appear in the spreadsheet for that token: the row containing the old time, time, oldF1, F1, oldF2, F2 information, and the row containing the information for its removal (either OK or removed). This second row doesn't contain measurement information, but contains NA for all values. This is fine because the first row takes care of this.
   • HOWEVER, when a change made is due to changing the max number of formants, this first row doesn't appear. Only one row appears after the token's been cleared, and all measurement values are NA. This means that the new F1 F2 values are not getting logged. maxForms is equal to the number of formants used in the remeasurement (if you use remeasureF but not if you use remeasureP, as you said Misha), but the F1 and F2 values are not logged.
   • Furthermore, this is the case for both remeasureF and remeasureP.
   • If both formants and the time is changed (e.g. if you change the number of formants in remeasureP and then also move the cursor before hitting Log1), both rows appear as they should. The maxNum of formants remains unchanged (again, as you said it would), but the important F1 and F2 values are accurately logged.
3. I don't think this is actually an issue, but just in case: the rows are not getting appended to the end of the file; they seem to be added in an order I can't figure out. most of my changes were logged chronologically, but some were not. For example, when I remeasure% and then click a new token and clear it, the row containing the cleared info appears a few rows up from the row containing the new measurement info.

[msonderegger]

Great -- thanks for the careful check, and thanks to Misha for fixing 2 and 3 above. I feel better already knowing there's only one real issue.

Thea, could you create an issue for #2, and Misha, could you prioritize this issue? (We do need the new F1 and F2 measurements when nFormants is changed, even if the new nFormants itself can't be logged.)

On #3: Misha, this could this be an issue if updateFormants.py is relying on knowing which row corresponding to a given token was written last... does it? (Or is chronological order no longer relevant now that the game is removing tokens?) It does seem preferable that rows are written out in the order changes were made, in terms of making the program usable for a wider range of applications. (On that note, the list of people I've shown plotmish to at this workshop who say "how can I get this for my data X, as soon as possible?" is growing..!)

[mishaschwartz]

I'm going to prioritize this and hopefully will have a fix by tomorrow.
updateFormants.py should be relying on the id number to change the formant values but there could be a bug that I'm not seeing. I'll double check and get back.
In terms making it more useful, I'm almost done a slimmed down and fully commented version that should be ready to release in a day or so. On that note though, what should I do about the celex issue that we discussed (that we can't distribute something that contains it)

[mishaschwartz]

The only time F1 and F2 should be something other that 'NA' in the log file is when they are changed. If they are removed, then these will be 'NA' because no change has been made. When updateFormants.py is run it will change only the vowels that have something other than 'NA' in the newF1 and newF2 columns. When updateFormants.py is run it will read down the file and make any changes as it goes. That way, if it sees a vowel has been changed it will change it and then if it sees that it's been removed later it will add the note that indicates that it's been removed.
That being said, there was a problem with the order that the log files were written (I had a momentary lapse and forgot that dictionaries are unordered) but that's been fixed now (problem 3 above).
In reality the only columns that are really necessary in the log files are the file name, the vowel id, new maxFormants, new time, new F1 and new F2. The rest are not touched by updateFormants.py.

As they are right now, the log files are designed simply as something for the updateFormants.py to read, not really something that you should be looking at for any info. I could, if you wanted, write the F1 and F2 values for the formants even when they're removed but that would make no difference to updateFormants.py. However this would be more readable if you want to be looking at the log files directly.

So the question is: Do we want to be examining the log files or just the corrected formant.txt files?

[msonderegger]

Thanks for this, and for the reminder that the log files aren't really meant to be readable. I think I suggested to Thea to use the log files for calibration with Maggie and Liam because I hadn't realized that updateFormants.py existed. It sounds like it's a better idea to just look at the corrected formant.txt files for calibration, compared to the log files? (Or better yet, the CSV that results from processing these files , but maybe that's not possible yet, unless formantsToCsv.py has been updated.) Whatever Thea thinks is best (Thea?)

The log files don't need to be very readable. But for debugging purposes (i.e., if someone adapts the code for their own project), it's good for them to be readable when the user is aided by a description in the readme (or even reading comments in the code). Having the lines be written in chronological order helps with this, so it's good you fixed that.

On Sat, May 31, 2014 at 5:09 PM, mishaschwartz notifications@github.com wrote:

The only time F1 and F2 should be something other that 'NA' in the log file is when they are changed. If they are removed, then these will be 'NA' because no change has been made. When updateFormants.py is run it will change only the vowels that have something other than 'NA' in the newF1 and newF2 columns. When updateFormants.py is run it will read down the file and make any changes as it goes. That way, if it sees a vowel has been changed it will change it and then if it sees that it's been removed later it will add the note that indicates that it's been removed.

That being said, there was a problem with the order that the log files were written (I had a momentary lapse and forgot that dictionaries are unordered) but that's been fixed now (problem 3 above).

In reality the only columns that are really necessary in the log files are the file name, the vowel id, new maxFormants, new time, new F1 and new F2. The rest are not touched by updateFormants.py.

As they are right now, the log files are designed simply as something for the updateFormants.py to read, not really something that you should be looking at for any info. I could, if you wanted, write the F1 and F2 values for the formants even when they're removed but that would make no difference to updateFormants.py. However this would be more readable if you want to be looking at the log files directly.

So the question is: Do we want to be examining the log files or just the corrected formant.txt files?

— Reply to this email directly or view it on GitHub https://github.com/mlml/plotmish/issues/18#issuecomment-44763914.

[msonderegger]

About the CELEX issue -- this is an annoying but important point. I think it'd be good to add (yet another) option to plotmish when it opens:

1. Use the Celex file, but the user specifies where it is (and put in the argparse help that it's called epw.cd (or whatever it is)). In this case, the right vowel panel is Celex vowels (as is it now)
2. Make the right vowel panel the Arpabet vowel for the full pronunciation of the word, and keep the left vowel panel the Arpabet vowel for the force-aligned pron (as it is now). We can distribute CMUDict, so this is fine.

2 will probably correspond to a more common use case anyway -- most users

will be using FAVE or P2FA, and relying on the vowel codes in Cmudict; the Celex vowels are a (crucial) specialization for analyzing British speech.

Sound good?

Morgan

On Fri, May 30, 2014 at 4:03 PM, mishaschwartz notifications@github.com wrote:

I'm going to prioritize this and hopefully will have a fix by tomorrow.

updateFormants.py should be relying on the id number to change the formant values but there could be a bug that I'm not seeing. I'll double check and get back.

In terms making it more useful, I'm almost done a slimmed down and fully commented version that should be ready to release in a day or so. On that note though, what should I do about the celex issue that we discussed (that we can't distribute something that contains it)

— Reply to this email directly or view it on GitHub https://github.com/mlml/plotmish/issues/18#issuecomment-44708757.

[thealk]

Good point about the files not being intended for calibration. I did know this, but I figured they'd be useful to go into a very small subset of a speaker's data (e.g. one vowel for one clip for one speaker, using a more specific keyword like Luke-0050). Since the csv files are organized by clip this seemed like the easiest way to do it. We'd initially calibrate by checking to see if we removed the same items for being unacceptable, then seeing if the items we okay'd were changed to similar values (and this was where we began running into difficulties with the output). I think now that the formant values are being output correctly this is still an okay way to calibrate. I do think the most useful thing would still be to be able to search for a token in plotmish by using its ID, start time, or word, though I don't think that's going to be necessary for this batch (just in the future if you want to train more people, and also if other users want to easily look up changes before sending them to updateFormants.py).