Ignore FORM tags in HTMLHighlighter

GoogleCodeExporter commented 9 years ago

When using HTMLHighlighter some times boilerpipe keeps some artifacts related 
coming from FORM and LABEL tags.

This can be easily prevented by addding a new ignorable element to TAG_ACTIONS 
map in HTMLHighlighter.java:

TAG_ACTIONS.put("FORM", TA_IGNORABLE_ELEMENT);

Original issue reported on code.google.com by xavi.beu...@gmail.com on 24 Mar 2012 at 6:40

GoogleCodeExporter commented 9 years ago

Issue 45 has been merged into this issue.

Original comment by ckkohl79 on 25 Mar 2012 at 2:12

GoogleCodeExporter commented 9 years ago

Adding FORM as an ignorable element at the highlighter (but not at the 
Extractor itself) has two disadvantages:

1. The highlighted HTML will not be consistent with the TextDocument's content 
information (i.e., it should rather go into DefaultTagActionMap)
2. FORM may span over many otherwise relevant text blocks. Adding FORM as 
TA_IGNORABLE_ELEMENT significantly reduces extraction accuracy (with L3S-GN1, 
avg. token-level F1 goes down by about 5% to 7%).

Could you please give examples (URLs) where boilerpipe fails currently?
Please try the very latest version from trunk, or -- preferably -- the version 
at http://boilerpipe-web.appspot.com/

Original comment by ckkohl79 on 25 Mar 2012 at 2:19

Added labels: Type-Enhancement
Removed labels: Type-Defect

GoogleCodeExporter commented 9 years ago

Adding FORM as an ignorable element at the highlighter (but not at the 
Extractor itself) has two disadvantages:

1. The highlighted HTML will not be consistent with the TextDocument's content 
information (i.e., it should rather go into DefaultTagActionMap)
2. FORM may span over many otherwise relevant text blocks. Adding FORM as 
TA_IGNORABLE_ELEMENT significantly reduces extraction accuracy (with L3S-GN1, 
avg. token-level F1 goes down by about 5% to 7%).

Could you please give examples (URLs) where boilerpipe fails currently?
Please try the very latest version from trunk, or -- preferably -- the version 
at http://boilerpipe-web.appspot.com/

Original comment by ckkohl79 on 25 Mar 2012 at 2:19

sriniavas / boilerpipe

Ignore FORM tags in HTMLHighlighter #44