search

ratul0 / boilerpipe

Automatically exported from code.google.com/p/boilerpipe
0 stars 0 forks source link

Title detection: Treat non-breaking space as whitespace #41

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago 
Character.isWhitespace(String) does not consider the non breaking space 
character (160) to be whitespace.  This causes whitespace to not be correctly 
trimmed when the non breaking space character is involved.  This can cause 
DocumentTitleMatchClassifier to miss a title match as well as other whitespace 
related problems.

The following article uses the character in the title and a few other places:
http://espn.go.com/dallas/nfl/story/_/id/7560381/do-anthony-spencer

Original issue reported on code.google.com by tucker...@gmail.com on 20 Mar 2012 at 2:58

Attachments:

GoogleCodeExporter commented 9 years ago 
Hi tuckerpmt,

thanks for reporting this. Especially the link to the article helped me 
figuring out what the problem was.
I have fixed this in r167, although in a slightly different way.

Cheers,
Christian

Original comment by ckkohl79 on 21 Mar 2012 at 9:10

GoogleCodeExporter commented 9 years ago 
Issue 38 has been merged into this issue.

Original comment by ckkohl79 on 21 Mar 2012 at 9:10

GoogleCodeExporter commented 9 years ago 
Issue 39 has been merged into this issue.

Original comment by ckkohl79 on 21 Mar 2012 at 9:10

GoogleCodeExporter commented 9 years ago 
Issue 42 has been merged into this issue.

Original comment by ckkohl79 on 21 Mar 2012 at 9:11

GoogleCodeExporter commented 9 years ago 
Issue 40 has been merged into this issue.

Original comment by ckkohl79 on 21 Mar 2012 at 9:11

GoogleCodeExporter commented 9 years ago 
Glad to help! :)

Though I am curious as to why you chose to scan all of the characters again 
instead of handle it where you currently are converting other whitespace 
characters to the standard space character?

Original comment by tucker...@gmail.com on 21 Mar 2012 at 10:44