herd does not set character encoding when serving js files

eclipse-archived / ceylon-herd

The Ceylon repository web application

Apache License 2.0

21 stars 11 forks source link

herd does not set character encoding when serving js files #243

Closed gavinking closed 8 years ago

gavinking commented 8 years ago

It seems to me that Herd doesn't explicitly set the character encoding of .js files to UTF-8. I think it should, right, @FroMage?

FroMage commented 8 years ago

Well, only if it knows what charset they were written in… And it doesn't. I can "pretend" they're utf8, but then that's a lie if anyone uploads stuff in another charset. And we can't really check that they compile stuff in that charset either. I think the proper thing to do is to store the encoding charset in the model file (JS compiler job) like we do for the binary verion, and have Herd store this info and serve the files properly.

That does not help for JS files it will serve as part of the module API docs, but we can't really fix the issue there since JS files do not reflect their encoding.

gavinking commented 8 years ago

Well then I think we need a rule that the compiler always generates the JS in UTF-8.

FroMage commented 8 years ago

We could look how node deals with encoding too. How does it load JS files from the FS? I assume using the platform encoding.

quintesse commented 8 years ago

Or perhaps they generate JS files with BOMs and use that?

FroMage commented 8 years ago

So, this is all sorts of wrongs, since:

We don't know if they are UTF-8
Some people say that the right MIME type for JS being application/javascript does not allow a charset (because application and not text), so it is ignored.
Apparently it's the source HTML charset that is applied to the JS files, or any specified script charset="" attribute
Requirejs forces the use of UTF-8 when in the browser, and will ignore the response headers
No clue what node.js does.

But I've added the damn header just to make you all shut the hell up ;)