ArkadiuszMichalski
commented
8 years ago @annevk can you write what should be done for this case when using some replacemenet labels?
[save using UTF-8 without BOM]
<!DOCTYPE html>
<html>
<head>
<meta charset="iso-2022-cn">
<title>Encoding test</title>
</head>
<body>
<script>
document.write("document.characterSet: " + document.characterSet + "<br>");
document.write("document.charset: " + document.charset + "<br>");
document.write("document.inputEncoding: " + document.inputEncoding);
</script>
</body>
</html>When I use labels for UTF-16LE/BE or x-user-defined then it's mapped to UTF-8 and windows-1252., but for replacement browsers (Firefox and Chrome) returns � as content and report using replacement encoding, should also be UTF-8 per getting an output encoding ?
Here you have online links to page that use document encoded by UTF-8 but use other meta (read encoding using right click and "View Page Info"): http://w3c-test.org/dom/nodes/encoding.py?label=utf-16be http://w3c-test.org/dom/nodes/encoding.py?label=utf-16le http://w3c-test.org/dom/nodes/encoding.py?label=x-user-defined
http://w3c-test.org/dom/nodes/encoding.py?label=csiso2022kr http://w3c-test.org/dom/nodes/encoding.py?label=hz-gb-2312 http://w3c-test.org/dom/nodes/encoding.py?label=iso-2022-cn http://w3c-test.org/dom/nodes/encoding.py?label=iso-2022-cn-ext http://w3c-test.org/dom/nodes/encoding.py?label=iso-2022-kr
And another question around HTML, which algorithm is last when finally using encoding, encoding sniffing algorithm or decode from encoding spec? Ask because I see this:
"Usually, the encoding sniffing algorithm defined below is used to determine the character encoding. Given a character encoding, the bytes in the input byte stream must be converted to characters for the tokenizer's input stream, by passing the input byte stream and character encoding to decode."
"The document's character encoding must immediately be set to the value returned from this algorithm, at the same time as the user agent uses the returned value to select the decoder to use for the input byte stream." << description for sniffing algorithm
So sniffing alghorithm only return encoding which is passing to decode algorithm (and this one make some correct) or finall encoding comes from sniffing? Basically I try figured out where excatly in HTML or Encoding spec document's encoding is set when input has BOM, like here:
[save using UTF-16LE with BOM]
<!DOCTYPE html>
<html>
<head>
<meta charset="iso-8859-2">
<title>Encoding test</title>
</head>
<body>
<script>
document.write("document.characterSet: " + document.characterSet + "<br>");
document.write("document.charset: " + document.charset + "<br>");
document.write("document.inputEncoding: " + document.inputEncoding);
</script>
</body>
</html>Browsers output UTF-16LE for all IDL attributes but I don't see prose where it's happend.
annevk
E.g., it expects "
WINDOWS-1252" which https://dom.spec.whatwg.org/#dom-document-characterset definitely does not define.