Title empty when parsing with TagSoup

[GoogleCodeExporter]

What steps will reproduce the problem?

I am using TagSoup for parsing HTML documents:

URL url = new URL("http://www.bbc.co.uk/news/uk-12038847")
Parser parser = new Parser();
BoilerpipeHTMLContentHandler handler = new BoilerpipeHTMLContentHandler();
parser.setContentHandler(handler);  
System.out.println("T: " + handler.toTextDocument().getTitle());
InputSource is = HTMLFetcher.fetch(url).toInputSource();
parser.parse(is);

What is the expected output? What do you see instead?

With the example document from the BBC you should get
"BBC News - Snow disrupts travel across northern Europe"
Instead it is null.

What version of the product are you using? On what operating system?

Trunk

Please provide any additional information below.

The problem can be fixed if I change 
BoilerpipeHTMLContentHandler.characters method
and move flushBlock() invocation from the begging of the method to its end (see 
attached patch). Since I have no idea why this helps, I am not sure if that is 
not braking other things.

Original issue reported on code.google.com by zzy...@gmail.com on 20 Dec 2010 at 8:02

Attachments:

• title.diff