search

dialogos-project / dialogos

The DialogOS dialog system.
https://www.dialogos.app
GNU General Public License v3.0
20 stars 7 forks source link

Set default charset more elegantly #208

Open alexanderkoller opened 4 years ago

alexanderkoller commented 4 years ago

Currently we get this warning at DialogOS startup:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.clt.dialogos.DialogOS (file:/Users/koller/Documents/workspace/dialogos/dialogos/Diamant/build/libs/Diamant-2.1.4-SNAPSHOT.jar) to field java.nio.charset.Charset.defaultCharset
WARNING: Please consider reporting this to the maintainers of com.clt.dialogos.DialogOS
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

This is because we force the default charset of the JVM to UTF-8 in DialogOS#enforceUtf8. At least at the time we wrote that code, this was needed to avoid bugs with MaryTTS grammars on Windows, where the default charset is CP1252.

The warning mentions that the way in which we abuse reflection for this will be denied in a future release, so let's keep an eye on this. Hopefully by that time this trick will be unnecessary because UTF-8 will be the cross-platform default encoding.

timobaumann commented 1 year ago

Our time has come: since Java 16, this is not allowed anymore and the linux I need to use has Java 17. To make things more interesting, it seems the JEP to make UTF-8 default has been implemented in Java 18 ( https://bugs.openjdk.org/browse/JDK-8187041 ).

timobaumann commented 1 year ago

There seem to be ways of re-enabling write access to java.nio.Charset.defaultCharset. In particular, we can add '--add-opens java.base/java.nio.charset=ALL-UNNAMED' to JvmArgs and it will work. However, once we start fiddling with JvmArgs anyway, we might as well set file.encoding there and merely throw a warning if it's not set to utf-8. 99.5% (as if there were more than 2 or 3...) of our users either start DialogOS via gradlew run or via the install4j startup scripts.

Any thoughts? (I'm a bit time-pressed to resolve this before our next round of high school students drops by to build amazing DialogOS systems...)

alexanderkoller commented 1 year ago

Ah, how interesting. Would it be a viable option to just assume that the default encoding is UTF-8 and ask Windows users to use Java 18? Do you happen to have a Windows machine at hand to test this?

timobaumann commented 1 year ago

I believe setting file.encoding from the commandline is the solution.

timobaumann commented 1 year ago

also note the following: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=4163515 ; we've been doing it wrong all along.