search

collective / collective.embeddedpage

collective.embeddedpage allows to embed remote HTML pages in Plone.
0 stars 4 forks source link

Encoding detection from request header #37

Open ZeEmEs opened 3 years ago

ZeEmEs commented 3 years ago

Character enocding detection with chardet.detect is not 100% reliable. Use instead character encoding from request header if available.

tisto commented 3 years ago

@ZeEmEs thanks for your PR! Since detecting character encoding is a very complicated matter I think we need a bunch of tests here to prove the problem and the solution. Would you be willing to work on this?

ZeEmEs commented 3 years ago

@tisto - as written it depend on the text given. Most of the time the detection is correct. If you need an text example, I think I can provide one. Overall the proposed change adds the Attempt reading the encoding from the request. On failing it falls back to the prvious behavior. So I don't think many changes are needed.