search

eclipse-archived / ceylon.formatter

A formatter for the Ceylon programming language, written in Ceylon.
Apache License 2.0
14 stars 11 forks source link

Generated sources encoding #60

Closed akberc closed 10 years ago

akberc commented 10 years ago

The generated sources encoding for characters ' and " may be incorrect UTF-8, at least on Windows.

lucaswerkmeister commented 10 years ago

I don't understand - quotes are regular ASCII characters, they're the same in almost every encoding.

What exactly was your problem? Did you run the formatter standalone or from the IDE?

akberc commented 10 years ago

From command line. Using Java 8 and Ant 1.9.4. On clean machines (both XP and Win7):

I will try to find the cause and see if I can fix it.

Sorry for the picture, no easy way to copy text from DOS Window: formatter-utf8

lucaswerkmeister commented 10 years ago

Oh, I see. Not a problem with formatted code, a compilation error! Because I like to use “this” and ‘these’ instead of "this" and 'these'.

IIRC the same problem popped up in ceylon-dist (by “Stéphane Épardaud”), and it can be fixed by declaring the encoding in the build.xml. Let me see…

lucaswerkmeister commented 10 years ago

Hm, we are declaring an encoding of UTF-8. Maybe the Ceylon compiler doesn’t tell javac the encoding?

Can you run ceylon compile and see if you still get errors?

lucaswerkmeister commented 10 years ago

Or (but that would be incredibly silly), could you try replacing “UTF-8” with “UTF8” in the build.xml?

sgalles commented 10 years ago

@lucaswerkmeister On Windows the generated files are not valid UTF8, it has nothing to do with Javac.

Actually the scheme you use to generate the files uses the default encoding of the OS, instead of UTF8. So on Windows you use CP1252. I know because : 1) I can make it work if i set set JAVA_TOOL_OPTIONS=-Dfile.encoding=UTF8 in my Windows shell 2) Without the workaround, the faulty quotes shows up as x92 in an UTF8 editor, and indeed this is invalid UTF8. In the attached screen shoot, this is how it looks like in notepad++ for instance encoding

lucaswerkmeister commented 10 years ago

Oh, I have to set the Writer encoding? Okay.

Can you try out if the fix60 branch works?

sgalles commented 10 years ago

Yes, indeed, fix60 branch works. Thanks !

lucaswerkmeister commented 10 years ago

Great, I pushed it to master. Thanks!

akberc commented 10 years ago

Thank you.

A minor note: ant clean does not remove the generated files in source\ceylon\formatter\options