joschan21 / profanity.dev

1.37k stars 104 forks source link

Normalise score #45

Open UtterDonkey opened 3 months ago

UtterDonkey commented 3 months ago

After running this on many different pieces of text, I got no score lower than 0.7106113. So naturally it would make sense to assume that the output is between 0.7 and 1. If the score were calculated by Math.max((vector.score - 0.7) / 0.3, 0), the score would be a number between 0 and 1, and a threshold of 0.85 using the original formula would be 0.5 (50%) using this adaptation, which makes more sense. The score is also clamped at 0, so if somehow the score was less than 0.7, the score outputted would never be negative. If you wanted 50% to be equivalent to the current 86% threshold, you would do Math.max((vector.score - 0.72) / 0.28, 0).

It may be useful to return this in a normalizedScore / normalisedScore from the API so as to not break anything currently using the score property.