Firefox 7.0.1 reports illegal character for static script content when content is UTF-8 with BOM.

[GoogleCodeExporter]

1) What's the problem?

Firefox 7.0.1 reports illegal character for static script content when content 
is UTF-8 with BOM.  

2) How can someone else reproduce it?

Save static js file with UTF-8 and BOM such that BOM is FFFE.

Firefox will report illegal character 

Firebug shows Response Content-Type as  
application/x-javascript; charset=WINDOWS-1252

Request Accept-Chars was ISO-8859-1,utf-8;q=0.7,*;q=0.7

3) What flavor of the Prudence are you using? Which version, and on what
operating system?

Prudence 1.1.1 revision 1604
Windows XP
FireFox 7.0.1

4) Please provide any additional information below:

IE8 and Chrome do not report any error.  I am curious how to send the response 
as UTF-8 (or whatever format the static resource was saved as) and not have the 
charset map to Windows...

Original issue reported on code.google.com by tgloc...@gmail.com on 3 Nov 2011 at 2:04

[GoogleCodeExporter]

Forgot to mention - using Savory Javascript version of Prudence.

Original comment by tgloc...@gmail.com on 3 Nov 2011 at 2:05

[GoogleCodeExporter]

Original comment by tal.liron on 3 Nov 2011 at 10:03

• Changed state: Started

[GoogleCodeExporter]

I wonder if it's a Restlet issue, or even a Jetty issue.

Could you possibly try using Grizzly instead of Jetty as the Restlet connector? 
(If you don't know how to do this, I can try.)

Original comment by tal.liron on 3 Nov 2011 at 10:19

[GoogleCodeExporter]

Also, could you please attach an example broken file to this bug report?

Original comment by tal.liron on 3 Nov 2011 at 10:20

[GoogleCodeExporter]

So try as I might, I cannot reproduce the error today...  

The Content-Type is still returned as application/x-javascript; 
charset=WINDOWS-1252 (rather than utf-8) 

Attached is an example file I was using - you would simply need to put it in 
/web/static/script/ (no need to try and use it) 

Note that the error was occurring for any file having BOM not just this one.  
At any rate, if you want to close the issue we can.

Original comment by tgloc...@gmail.com on 4 Nov 2011 at 9:06

Attachments:

• TGXMenu_v1.0.js

[GoogleCodeExporter]

Hm. Are you 100% sure the file was served through Prudence, and not through 
Apache or some other layer in front of Prudence?

Original comment by tal.liron on 6 Nov 2011 at 8:28

[GoogleCodeExporter]

Update, aspects of this issue came back to haunt.  Currently running FireFox 9 
and it does not report any error even though the illegal character  were 
there.  Prudence/jetty version remains unchanged. 

Did some digging and leared that Jetty respects the java -Dfile.encoding param.

Added -Dfile.encoding=UTF-8 ^ to the run.bat 

And now CSS files in static folder are returned with charset=UTF-8 

Hope this might help someone down the road!

Original comment by tgloc...@gmail.com on 13 Jan 2012 at 7:21

[GoogleCodeExporter]

Good work! This is a good start to investigate what the problem might be.

Original comment by emblempa...@gmail.com on 14 Jan 2012 at 12:15

[GoogleCodeExporter]

I've done some research, and this indeed is a widespread problem across many 
projects. Different OSes set the default charset differently, which is why I 
was unable to reproduce the bug. Apparently, it's especially troublesome on 
Mac, which uses a non-standard encoding.

I've patched the scripts for Prudence 2.0 to set file.encoding to utf8.

Also, to preempt the problem for dynamic files, I have patched the way 
Scripturian loads source code files to *always* use UTF8, so that it would be 
consistent across operating systems. I've also switched to using memory mapped 
files via nio, which would provide a tiny performance boost. (I actually know 
about this problem very well, but this code in Scripturian is quite old, from 
before I was smarter about it.)

Original comment by tal.liron on 14 Jan 2012 at 10:12

• Changed state: Fixed