u7122029 / text_gen_calibration

0 stars 0 forks source link

Add FrequencyTokenRemover Calibrator #20

Closed u7122029 closed 2 months ago

u7122029 commented 2 months ago

Works just like FrequencyTS, but removes the tokens that scored above the threshold instead of rescaling them via a temperature parameter.

This calibrator will be used in the motivation section to justify the intuition behind the metric.