search

VizierDB / vizier-scala

The Vizier kernel-free notebook programming environment
Other
34 stars 11 forks source link

Unable to run LOAD DATA cell on Java 18 #254

Open okennedy opened 1 year ago

okennedy commented 1 year ago

Describe the bug Running the LOAD DATA cell produces a java permissions error

To Reproduce

  1. Install and enable Java 18 or higher
  2. Run vizier and create any load dataset
  3. See the kryo field serializer error.

Also, the following test should pass (without kryo errors)

mill vizier.test.testOnly info.vizierdb.commands.DataCommandsSpec -- ex 'load, unload, and query'

Expected behavior The load dataset cell runs

Notes This appears to be related to the new Java module permissions system introduced in Java 9. Unnamed modules can no longer access hidden state in java.nio. The following command flags grant permissions to all legacy classes (I'm sure there's a security hole here, but not one that wasn't already present in Java 8).

        "--add-exports", "java.base/sun.nio.ch=ALL-UNNAMED",
        "--add-opens", "java.base/sun.nio.ch=ALL-UNNAMED",

These flags are already in forkArgs, but our JDK version detection logic in the millfile appears to break on Java 18 and 20. I've temporarily updated the millfile to default to JDK 11+ mode when it can't figure out the version... but it's possible this might break Java8.

okennedy commented 11 months ago

Limiting support to Java 11 until v2.1. Will upgrade to a newer spark at that point.