For people hoping to use this for AMT, it would be useful to have some code which, given a transcription and a ground truth, will output the proportion of errors which could be assigned to each degradation.
This would allow users to create a dataset for their specific use case.
We could also have it output recommended parameter values.
Or, ideally, output a json file directly which would be readable by the make_dataset script.
Some difficulties:
It won't be possible to get the distribution exactly correct, as some errors might be ambiguous (is it a pitch shift and a time shift? Or an add note and a remove note?)
We may want to allow the user to specify a window size for this. Then, we can also measure how many windows have no degradation. (#31)
We may want to allow multiple degradations per excerpt in some cases, eventually. (#32)
For people hoping to use this for AMT, it would be useful to have some code which, given a transcription and a ground truth, will output the proportion of errors which could be assigned to each degradation.
This would allow users to create a dataset for their specific use case.
We could also have it output recommended parameter values.
Or, ideally, output a json file directly which would be readable by the make_dataset script.
Some difficulties: