Use perplexity in metadata confidence return to make values more comparable across difference sentence lengths

coqui-ai / STT

🐸STT - The deep learning toolkit for Speech-to-Text. Training and deploying STT models has never been so easy.

https://coqui.ai

Mozilla Public License 2.0

2.23k stars 267 forks source link

Use perplexity in metadata confidence return to make values more comparable across difference sentence lengths #1857

Open reuben opened 3 years ago

reuben commented 3 years ago

Suggested by @bernardohenz:

beta_total = beta_weight * num_words
total_confidence = (confidence -  beta_total) / alpha_weight     # Discount weights from confidence (we're using deepspeech 0.7.1)

raw_probability = math.pow( math.e, total_confidence)  # Transform logarithm to probability
perplexity = math.pow(1/raw_probability, 1/num_words)  # Compute perplexity

bernardohenz commented 3 years ago

Just posting references to support this (ref1, ref2)