Clean up user-submitted HTML, preserving whitelisted elements and whitelisted attributes on a per-element basis. Built on htmlparser2 for speed and tolerance
MIT License
3.68k
stars
349
forks
source link
[Question] Best practice to cleanup HTML Spaghetti code #630
It's unclear what you're trying to do exactly, but sanitize-html is quite good at keeping only the tags and attributes you approve, as you can see in the documentation. If you want to do more subtle things, there are transformation features. If your needs exceed that, then you might consider using sanitize-html as a first pass and then cheerio for the transformations.
I came here to report a similar issue. An unclosed attribute (missing final double-quote) will cause everything from the start of that tag through to the end of the input to be stripped by sanitize-html.
// ↓ Missing double-quote
sanitize(`Hello, world. <a href="/this>this</a> is a demo of this behavior. <b>I won't be in the output!</b>`)
// => 'Hello, world. '
Angle brackets are not forbidden in quoted HTML attributes, and in fact this document produces the expected title on hover in Chrome:
<h4 title="this is a title<containing><punctuation>">h4 body</h4>
If both the standard and actual browsers permit it then sanitize-html can't reliably detect that it is "wrong" (because it isn't, strictly speaking). Also this behavior is coming from the htmlparse2 module in any case, but keep in mind it is not a bug before reporting anything there.
@boutell Got it. So in the case of invalid HTML (the double quote never closes anywhere), is there any way to get an error back instead of having large portions of the input stripped out?
If you mean at the very end of the fragment, when you're absolutely sure no
closing quote is coming, it looks like htmlparser2 always tidies up at the
end by closing whatever isn't closed, and we're downstream of that. There
may or may not be htmlparser2 options that modify this behavior.
On Wed, Oct 4, 2023 at 5:16 PM Corey Ward @.***> wrote:
@boutell https://github.com/boutell Got it. So in the case of invalid
HTML (the double quote never closes anywhere), is there any way to get an
error back instead of having large portions of the input stripped out?
Question or comment
I need to cleanup a random HTML page to extract readable content. Modern website use A LOT of Spaghetti HTML. For instance :
<div> <div><div> <a href="/"> <div>Title of the site</div> </a> <div lazyload="event"><!--lazy <div class="headerPageHtml"><a href="/include/news.xml" target="_blank"><img id="socialRss" alt="access to rss" src="/asset/social/rss.png"></a></div></div></div>In this example I don't want all the
<div>and the formating<div><img></div>Should I strip all theIt's unclear what you're trying to do exactly, but sanitize-html is quite good at keeping only the tags and attributes you approve, as you can see in the documentation. If you want to do more subtle things, there are transformation features. If your needs exceed that, then you might consider using
sanitize-htmlas a first pass and thencheeriofor the transformations.I came here to report a similar issue. An unclosed attribute (missing final double-quote) will cause everything from the start of that tag through to the end of the input to be stripped by
sanitize-html.Angle brackets are not forbidden in quoted HTML attributes, and in fact this document produces the expected title on hover in Chrome:
If both the standard and actual browsers permit it then sanitize-html can't reliably detect that it is "wrong" (because it isn't, strictly speaking). Also this behavior is coming from the
htmlparse2module in any case, but keep in mind it is not a bug before reporting anything there.@boutell Got it. So in the case of invalid HTML (the double quote never closes anywhere), is there any way to get an error back instead of having large portions of the input stripped out?
If you mean at the very end of the fragment, when you're absolutely sure no closing quote is coming, it looks like htmlparser2 always tidies up at the end by closing whatever isn't closed, and we're downstream of that. There may or may not be htmlparser2 options that modify this behavior.
On Wed, Oct 4, 2023 at 5:16 PM Corey Ward @.***> wrote:
Okay, I think we can live with that for now. Thank you!