search

enso-org / enso

Hybrid visual and textual functional programming.
https://enso.org
Apache License 2.0
7.31k stars 317 forks source link

Support for --jvm option in Enso runner #10374

Open JaroslavTulach opened 6 days ago

JaroslavTulach commented 6 days ago

Pull Request Description

Addresses one of two concerns of #5298 - adds support for --jvm argument to allow us to switch from native image built Enso binary (as developed by #10126) to regular JVM based Enso execution. This change doesn't affect production builds. The native executable continues to be only built by engine-runner/buildNativeImage which is tested on CI, but not in the production jobs.

Typical Usecase

With e67cc87 one can build:

sbt:enso> buildEngineDistribution
sbt:enso> engine-runner/buildNativeImage

and then the bin/enso launcher is going to be a native executable (of a huge size):

enso$ ls built-distribution/enso-engine-*linux-*/enso-*/bin/enso -hl
278M

The bin/enso file is native executable with built in support for executing Standard.Base library (because of #10296). However, should there be an Enso file short.enso like

from Standard.Base import IO

polyglot java import java.lang.Short

main =
    IO.println <| Short.valueOf "445"

then executing it with prebuilt native executable fails because it imports some class not used by Standard.Base and thus not compiled into the executable:

enso$ ./built-distribution/enso-engine-*linux-*/enso-*/bin/enso --run short.enso 
Execution finished with an error: Method `valueOf` of type java.lang.Short could not be found.
        at <enso> hi.main<arg-1>(hi.enso:6:19-37)
        at <enso> hi.main(hi.enso:6:5-37)

at that moment one can use the --jvm attribute and then the execution succeeds:

enso$ ./built-distribution/enso-engine-*linux-*/enso-*/bin/enso --jvm --run /home/devel/hi.enso 
445

Native executable gives us instant startup. The --jvm switch gives us compatibility with the JVM mode.

Checklist

Please ensure that the following checklist has been satisfied before submitting the PR:

enso-bot[bot] commented 6 days ago

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-26):

Progress: - create PR for the --jvm argument: https://github.com/enso-org/enso/pull/10374

GitHub
Support for --jvm option in Enso launcher by JaroslavTulach · Pull Request #10374 · enso-org/enso
Pull Request Description Addresses one of two concerns of #5298 - adds support for --jvm argument to allow us to switch from native image built Enso binary (as developed by #10126) to regular JVM b...
Discord
Discord - Group Chat That’s All Fun & Games
Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.
JaroslavTulach commented 5 days ago

The current status. enso or enso.exe is being generated into ./built-distribution/enso-engine-*/enso-*/bin/ directory on all operating systems Linux, Mac and Windows.

Why I run a simple program via ./bin/enso or .\bin\enso.exe I get:

java.lang.UnsupportedOperationException: Preview Features not enabled, need to run with --enable-preview
        at java.base@21.0.2/jdk.internal.misc.PreviewFeatures.ensureEnabled(PreviewFeatures.java:49)
        at java.base@21.0.2/java.lang.Thread.ofVirtualWithoutLoom(Thread.java:680)
        at java.base@21.0.2/java.util.concurrent.Executors.newVirtualThreadPerTaskExecutor(Executors.java:269)
        at org.enso.downloader.http.HTTPDownload$.asyncDownload(HTTPDownload.scala:82)
        at org.enso.downloader.http.HTTPDownload$.fetchString(HTTPDownload.scala:53)
        at org.enso.editions.updater.EditionUpdater.$anonfun$downloadEditionRepositoryManifest$1(EditionUpdater.scala:70)
        at scala.util.Try$.apply(Try.scala:210)
        at org.enso.editions.updater.EditionUpdater.downloadEditionRepositoryManifest(EditionUpdater.scala:67)
        at org.enso.editions.updater.EditionUpdater.$anonfun$fetchManifests$4(EditionUpdater.scala:106)

what can be wrong?

When I use the --jvm option, bin\enso.exe.exe --jvm --run f.enso then it cannot find libraries:

[TRACE] [2024-06-27T09:51:40.293] [enso.org.enso.interpreter.EnsoLanguage] Detected text mode, using a standalone lock manager.
[DEBUG] [2024-06-27T09:51:40.293] [org.enso.distribution.DistributionManager] Detected paths: DistributionPaths(
  dataRoot = C:\Users\Jaroslav Tulach\AppData\Local\enso,
  runtimes = C:\Users\Jaroslav Tulach\AppData\Local\enso\runtime,
  engines  = C:\Users\Jaroslav Tulach\AppData\Local\enso\dist,
  bundle   = None,
  config   = C:\Users\Jaroslav Tulach\AppData\Local\enso\config,
  locks    = C:\Users\Jaroslav Tulach\AppData\Local\enso\lock,
  logs     = C:\Users\Jaroslav Tulach\AppData\Local\enso\log,
  tmp      = C:\Users\Jaroslav Tulach\AppData\Local\enso\tmp,
  profiling = C:\Users\Jaroslav Tulach\AppData\Local\enso\profiling,
  ensoHome  = C:\Users\Jaroslav Tulach\enso,
  customEditions = List(C:\Users\Jaroslav Tulach\enso\editions),
  localLibrariesSearchpaths = List(C:\Users\Jaroslav Tulach\enso\libraries)
)
[DEBUG] [2024-06-27T09:51:40.464] [org.enso.distribution.config.GlobalConfigurationManager] Global config [C:\Users\Jaroslav Tulach\AppData\Local\enso\config\global-config.yaml] 
not found, falling back to defaults.
[TRACE] [2024-06-27T09:51:40.664] [org.enso.librarymanager.LibraryLocations] Local library search paths = List(C:\Users\Jaroslav Tulach\enso\libraries)
[TRACE] [2024-06-27T09:51:40.664] [org.enso.librarymanager.LibraryLocations] Primary library cache = C:\Users\Jaroslav Tulach\AppData\Local\enso\lib
[TRACE] [2024-06-27T09:51:40.664] [org.enso.librarymanager.LibraryLocations] Auxiliary (bundled) library caches = List()
[DEBUG] [2024-06-27T09:51:40.737] [enso.org.enso.compiler.Compiler] Initialising IR for [Standard.Builtins.Main].
[TRACE] [2024-06-27T09:51:40.842] [enso.org.enso.interpreter.caches.Cache] Loaded cache for Standard.Builtins.Main with 222 bytes in 93 ms
[TRACE] [2024-06-27T09:51:40.843] [enso.org.enso.interpreter.caches.Cache] Using cache for [Standard.Builtins.Main at location [C:\Users\Jaroslav Tulach\AppData\Local\enso\cache\ir\Standard\Builtins\0.0.0-dev\0.0.0-dev\Standard\Builtins].
[DEBUG] [2024-06-27T09:51:40.844] [enso.org.enso.interpreter.runtime.SerializationPool] Restored IR from cache for module [Standard.Builtins.Main] at stage [AFTER_STATIC_PASSES].[DEBUG] [2024-06-27T09:51:40.846] [enso.org.enso.interpreter.runtime.SerializationPool] Deserializing module Standard.Builtins.Main from IR file: true
[DEBUG] [2024-06-27T09:51:40.848] [enso.org.enso.compiler.Compiler] Parsing module [f].
[DEBUG] [2024-06-27T09:51:40.848] [enso.org.enso.interpreter.runtime.SerializationPool] Unable to load a cache for module [f].
[DEBUG] [2024-06-27T09:51:40.853] [enso.org.enso.interpreter.runtime.SerializationPool] Deserializing module f from IR file: false
[DEBUG] [2024-06-27T09:51:40.853] [enso.org.enso.compiler.Compiler] Loading module [f] from source.
[TRACE] [2024-06-27T09:51:40.897] [org.enso.interpreter.runtime.DefaultPackageRepository] Resolving library Standard.Base.
[TRACE] [2024-06-27T09:51:40.897] [org.enso.librarymanager.DefaultLibraryProvider] Resolved Standard.Base to [Right(0.0.0-dev)].
[TRACE] [2024-06-27T09:51:40.913] [org.enso.librarymanager.published.DefaultPublishedLibraryProvider] Standard.Base was not found in any caches, it will need to be downloaded.   
[TRACE] [2024-06-27T09:51:40.913] [org.enso.librarymanager.published.cache.DownloadingLibraryCache] Trying to install [Standard.Base:0.0.0-dev].
[DEBUG] [2024-06-27T09:51:40.928] [org.enso.downloader.http.HTTPDownload] Fetching [https://libraries.release.enso.org/libraries/Standard/Base/0.0.0-dev/manifest.yaml].
[INFO] [2024-06-27T09:51:41.061] [org.enso.interpreter.instrument.NotificationHandler$TextMode] Downloading library manifest of [Standard.Base].
[WARN] [2024-06-27T09:51:41.642] [org.enso.interpreter.runtime.DefaultPackageRepository] Resolution failed with [The package download has failed: The server has responded with 404 status.].
C:\enso\f.enso:1:1: error: Package containing the module Standard.Base.Main could not be loaded: The package download has failed: The server has responded with 404 status.
    1 | from Standard.Base import all
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
C:\enso\f.enso:4:5: error: The name `File` could not be found.
    4 |     File.new "C:\dev:a"
      |     ^~~~
Aborting due to 2 errors and 0 warnings.
org.graalvm.polyglot.PolyglotException: C:\enso\f.enso:1:1: error: Package containing the module Standard.Base.Main could not be loaded: The package download has failed: The server has responded with 404 status.
    1 | from Standard.Base import all
      | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~

These two results are from Windows and Mac - they both work on Linux - I probably have proper symlink setup there since I used it daily. But we want to get this working out of the box, without any setup, right? Anyway right now we are back in a mode when the Enso runtime tries to download something...

JaroslavTulach commented 5 days ago

Downloading Enso Bundle Linux to understand the layout of such a distribution. Áááá:

$ tar fxvz ~/Stažené/enso-bundle-2024.2.1-nightly.2024.6.24-linux-amd64.tar.gz
$ ./enso/bin/enso

isn't the engine-runner, but the launcher. Ach. ./enso/dist/2024.2.1-nightly.2024.6.24/bin/enso is the engine-runner! The layout of the Enso Bundle comes in Portable Enso Distribution Layout .

enso-bot[bot] commented 4 days ago

Jaroslav Tulach reports a new STANDUP for yesterday (2024-06-27):

Progress: - integrated: https://github.com/enso-org/enso/pull/10380

Discord
Discord - Group Chat That’s All Fun & Games
Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.
Discord
Discord - Group Chat That’s All Fun & Games
Discord is great for playing games and chilling with friends, or even building a worldwide community. Customize your own space to talk, play, and hang out.