search

NoahCarnahan / plagcomps

Other
6 stars 1 forks source link

punctuation % is broken #6

Closed nrjones8 closed 10 years ago

nrjones8 commented 10 years ago

when running controller on the first file in our training set, we're missing punctuation. the file:

/copyCats/pan-plagiarism-corpus-2009/intrinsic-detection-corpus/suspicious-documents/part1/suspicious-document00536.txt

specifically misses the ";" in the following: "have already been acted upon by the House of Representatives; the bill"

and a number of others.

nrjones8 commented 10 years ago

Looks like it's an issue with our indexing; some places we're indexing by character, some places we're indexing by sentence, some places by word etc.