jshost will not load script files saved in any format other than ANSI

[GoogleCodeExporter]

To reproduce:
=============

1) Open Notepad
2) Enter:

LoadModule('jsstd');
Print('Works');

3) Save with the encoding set to something other than ANSI
4) run jshost [filename]

Problem output:
===============

For Unicode:

C:\Stuff\devel\seq>jshost unicode.js
unicode.js:1: SyntaxError: illegal character:
unicode.js:1: ■L
unicode.js:1: ..^
Unable to compile the script unicode.js.

For Unicode big-endian:

C:\Stuff\devel\seq>jshost unicodebe.js
unicodebe.js:1: SyntaxError: illegal character:
unicodebe.js:1: ■ 
unicodebe.js:1: ..^
Unable to compile the script unicodebe.js.

For UTF-8:

C:\Stuff\devel\seq>jshost utf8.js
utf8.js:1: SyntaxError: illegal character:
utf8.js:1: ´╗┐LoadModule('jsstd');
utf8.js:1: .^
Unable to compile the script utf8.js.

Expected output:
================

C:\Stuff\devel\seq>jshost ansi.js
Works

Comments:
=========

This makes it impossible to include Unicode glyphs in JS programs. We
cannot localise JS programs in any language that uses Unicode (such as
Japanese).

Presumably the JS interpreter is seeing the Byte-Order Mark at the
beginning of the Unicode file.

BOMs are:

FF FE  UCS-2LE or UTF-16LE
FE FF  UCS-2BE or UTF-16BE
EF BB BF  UTF-8

Clearly, JSlibs should support unicode files, which could include comments,
variable names or strings which use unicode characters.

Environment:
============

Notepad from Windows XP Build 2600.xpsp.080413 : Service Pack 3

C:\Stuff\devel\seq>jshost -version
Version r2916 / JavaScript-C 1.8.0 pre-release 1 2007-10-03

Original issue reported on code.google.com by phil_rho...@rocketmail.com on 21 Nov 2009 at 4:23

[GoogleCodeExporter]

Original comment by sou...@gmail.com on 21 Nov 2009 at 11:55

• Changed state: Accepted
• Added labels: Priority-High
• Removed labels: Priority-Medium

[GoogleCodeExporter]

- see commit http://code.google.com/p/jslibs/source/detail?r=2945

Original comment by sou...@gmail.com on 21 Nov 2009 at 5:26

• Changed state: Fixed