search

u7122029 / text_gen_calibration

0 stars 0 forks source link

Stopword and Punctuation removed dataset methods. #15

Closed u7122029 closed 2 months ago

u7122029 commented 2 months ago

Removing stopwords and punctuation is known to improve accuracy for some NLP tasks. Will we see the same results here?

More importantly, how does removing stopwords and punctuation affect calibration?