RFW0111: Prioritise task for Final Review based on CER against Model Inference

[spsither]

RFW0111: Prioritise task for Final Review based on CER against Model Inference

Summary

For the Final Reviewer, we can give them tasks with more errors before other tasks.

Key Concepts

CER: Character Error Rate is a metric we use for measuring distance between two string Final Review: This is a team that does quality control and reviews the team leads.

Context

Currently, the Final Reviewer works on tasks in the order of the date reviewed. A more sensible ordering might give us a better payoff.

Compare the inference_transcript column and the reviewed_transcript to get CER and use that CER to sort the task for the Final Reviewer.

Outputs

We can use the time of the Final Reviewer more efficiently by focusing their time on transcriptions with more issues.

Inputs

• All the tasks we have reviewed once.
• STT Model inference to compare the transcription.

Timeline

Jan 30th

References

CER

[kaldan007]

can u mention the filtering strategy briefly in context and update the output accordingly

[tenkus47]

can we mention the minimum CER for which a data should be sent for reviewing or is it just for sorting purpose

[spsither]

We can sort. So they'll work on tasks with more errors first but if they have more time they'll eventually fix smaller issues as well.