search

AbeHandler / rookie

The Rookie Text Analysis System
10 stars 1 forks source link

remove bylines in lens preprocessing #187

Closed AbeHandler closed 8 years ago

AbeHandler commented 8 years ago

this is a quirk of the lens corpus, but bylines are included there. you may want to just exclude anything at the top of the story with a “by X Y” or you can ignore it and assume other corpuses won’t have this.

AbeHandler commented 8 years ago

Going to hold off on this for now until when and if we need a clean version of lens for use study