Improve alignment and error messaging/debugging

[roedoejet]

This are a few ideas following a meeting between myself, @joanise and @littell .

The problem is that, currently, when alignment fails, it's not clear exactly why the alignment failed. It's often due to errors in transcription, or false starts from the speaker. This issue is to improve ReadAlong-Studio's ability to handle these occurrences, but also to provide helpful insight as to where in the document the alignment is least certain.

Some possible things to add

• [ ] Watch log perplexity of certain transitions and record any sudden increases
• [ ] Add low-probability transitions for silence or 'garbage'. this is seemingly happening
• [ ] Dynamic adjustment of beam - strict at first, but relax after certain threshold
• [ ] Anomaly detection with outliers from the duration model

We should also be thinking about how to interact with the user when these issues come up. There should be a view in the ReadAlong-Studio web app for debugging that highlights areas where the beam was adjusted or where sudden changes of log perplexity occurred.

[joanise]

We will clearly never do this. There were nice ideas, but...

[roedoejet]

We will clearly never do this. There were nice ideas, but...

I mean don't we already do a dynamic adjustment of the beam? That's what the fallback mechanism in studio web is doing. I agree we probably don't need to do the other stuff at this point.