inexorabletash / text-encoding

Polyfill for the Encoding Living Standard's API
Other
720 stars 267 forks source link

TypeError: The code point 279 could not be encoded #75

Closed dciccale closed 6 years ago

dciccale commented 6 years ago

There is a character that fails to be encoded.

Here is how to reproduce the error:

const {TextEncoder} = require('./encoding');
const textEncoder = new TextEncoder('windows-1252', {NONSTANDARD_allowLegacyEncoding: true});
textEncoder.encode('ė');

Output:

.../encoding.js:285
    throw TypeError('The code point ' + code_point + ' could not be encoded.');
    ^

TypeError: The code point 279 could not be encoded.
    at encoderError (.../encoding.js:285:11)
    at SingleByteEncoder.handler (.../encoding.js:1622:9)
    at TextEncoder.encode (.../encoding.js:1299:30)
    at Object.<anonymous> (.../test-enc.js:4:13)
    at Module._compile (module.js:635:30)
    at Object.Module._extensions..js (module.js:646:10)
    at Module.load (module.js:554:32)
    at tryModuleLoad (module.js:497:12)
    at Function.Module._load (module.js:489:3)
    at Function.Module.runMain (module.js:676:10)

UPDATE: It only works with encoding windows-1257. Should I use that encoding instead?

inexorabletash commented 6 years ago

You should use UTF-8. Legacy encodings cover only some parts of some scripts. It's why Unicode was invented - an encoding that covers all scripts.

The legacy encodings should only be used when absolutely necessary for interop with systems where no other choice is available.

dciccale commented 6 years ago

That's exactly the reason why I have to use legacy encoding. thanks

On Nov 25, 2017 18:58, "Joshua Bell" notifications@github.com wrote:

You should use UTF-8. Legacy encodings cover only some parts of some scripts. It's why Unicode was invented - an encoding that covers all scripts.

The legacy encodings should only be used when absolutely necessary for interop with systems where no other choice is available.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/inexorabletash/text-encoding/issues/75#issuecomment-346955927, or mute the thread https://github.com/notifications/unsubscribe-auth/AAg7miTdFD2_FmE8Ud4DAxqv0i-iSecaks5s6FVMgaJpZM4QqW2A .