enso-org / enso

Hybrid visual and textual functional programming.
https://enso.org
Apache License 2.0
7.31k stars 317 forks source link

Integrate our launcher with Truffle launcher #5298

Open wdanilo opened 1 year ago

wdanilo commented 1 year ago

This task is automatically imported from the old Task Issue Board and it was originally created by Pavel Marek. Original issue is here.


Truffle launchers provide many useful capabilities, like option categories.

Our org.enso.runner.Main should not use org.apache.commons.cli at all, but rather truffle options.

By using Truffle launcher, we will get these advantages:

For example, js --help:

Usage: js [OPTION]... [FILE]...
Run JavaScript FILEs on the Graal.js engine. Run an interactive JavaScript shell if no FILE nor --eval is specified.

Arguments that are mandatory for long options are also mandatory for short options.

Basic Options:
  -e, --eval CODE       evaluate the code
  -f, --file FILE       load script file
  --module FILE         load module file
  --syntax-extensions   enable non-spec syntax extensions
  --print-result        print the return value of each FILE
  --scripting           enable scripting features (Nashorn compatibility option)
  --strict              run in strict mode
  --version             print the version and exit
  --show-version        print the version and continue

Runtime options:
  --polyglot                                   Run with all other guest languages accessible.
  --native                                     Run using the native launcher with limited access to Java libraries
                                               (default).
  --jvm                                        Run on the Java Virtual Machine with access to Java libraries.
  --vm.[option]                                Pass options to the host VM. To see available options, use '--help:vm'.
  --log.file=<String>                          Redirect guest languages logging into a given file.
  --log.[logger].level=<String>                Set language log level to OFF, SEVERE, WARNING, INFO, CONFIG, FINE,
                                               FINER, FINEST or ALL.
  --help                                       Print this help message.
  --help:vm                                    Print options for the host VM.
  --help:engine                                Print engine options.
  --help:all                                   Print all options.
  --version:graalvm                            Print GraalVM version information and exit.
  --show-version:graalvm                       Print GraalVM version information and continue execution.

Languages:
  [id]        [name]                  [website]
  js          JavaScript              https://www.graalvm.org/javascript/

Tools:
  [id]        [name]                  [website]
  agentscript Agent Script            
  coverage    Code Coverage           https://www.graalvm.org/tools/code-coverage/
  cpusampler  CPU Sampler             https://www.graalvm.org/tools/profiling/
  cputracer   CPU Tracer              https://www.graalvm.org/tools/profiling/
  dap         Debug Protocol Server   https://www.graalvm.org/tools/dap/
  heap        Heap Dump               
  heapmonitor Heap Allocation Monitor 
  insight     Insight                 https://www.graalvm.org/tools/graalvm-insight/
  inspect     Chrome Inspector        https://www.graalvm.org/tools/chrome-debugger/
  lsp         Language Server         https://www.graalvm.org/tools/lsp/
  memtracer   Memory Tracer           https://www.graalvm.org/tools/profiling/

  Use --help:[id] for component options.

See http://www.graalvm.org for more information.

Related issues:

Comments:

Btw. just a 'historical' note that keeping the runner Main very simple and rather rudimentary was a conscious decision made in the past - from the beginning this meant to be an internal entry point, not really exposed to the end-users. The end users are supposed to use the Enso Launcher which has a much nicer CLI. I'm not sure why it has not been more widely adopted by our devs.

I'm not at all against integrating this, even on the contrary - it would be great to be able to gu install enso at some point! But note that the 'official' launcher is a separate project and I want to ensure we do not duplicate too much stuff between the two - to avoid maintaining two functionalities doing the same thing - unless a decision comes to retire the old launcher. (Radosław Waśko - Jan 24, 2023)


JaroslavTulach commented 2 months ago

The work on

enables this task to be easily prototyped on top of https://github.com/enso-org/enso/pull/9810/commits/ece87469179d10635051dfcbe571a6436548bd26. With following changes:

diff --git build.sbt build.sbt
index 74ce47e86e..4130e1bb12 100644
--- build.sbt
+++ build.sbt
@@ -2092,7 +2092,7 @@ lazy val `runtime-fat-jar` =
     .settings(
       libraryDependencies ++= {
         val graalMods =
-          GraalVM.modules.map(_.withConfigurations(Some(Runtime.name)))
+          GraalVM.modules.map(_.withConfigurations(Some(Compile.name)))
         val langMods =
           GraalVM.langsPkgs.map(_.withConfigurations(Some(Runtime.name)))
         val logbackMods =
@@ -2203,6 +2203,7 @@ lazy val `engine-runner` = project
     commands += WithDebugCommand.withDebug,
     inConfig(Compile)(truffleRunOptionsSettings),
     libraryDependencies ++= Seq(
+      "org.graalvm.sdk"         % "launcher-common"         % graalMavenPackagesVersion,
       "org.graalvm.polyglot"    % "polyglot"                % graalMavenPackagesVersion,
       "org.graalvm.sdk"         % "polyglot-tck"            % graalMavenPackagesVersion % Provided,
       "commons-cli"             % "commons-cli"             % commonsCliVersion,
diff --git engine/runner/src/main/java/org/enso/runner/Main.java engine/runner/src/main/java/org/enso/runner/Main.java
index 2de741a519..81ea7c239c 100644
--- engine/runner/src/main/java/org/enso/runner/Main.java
+++ engine/runner/src/main/java/org/enso/runner/Main.java
@@ -14,6 +14,8 @@ import java.util.Arrays;
 import java.util.Collections;
 import java.util.HashMap;
 import java.util.LinkedList;
+import java.util.List;
+import java.util.Map;
 import java.util.UUID;
 import java.util.concurrent.Callable;
 import java.util.concurrent.TimeUnit;
@@ -44,6 +46,8 @@ import org.enso.polyglot.PolyglotContext;
 import org.enso.profiling.sampler.NoopSampler;
 import org.enso.profiling.sampler.OutputStreamSampler;
 import org.enso.version.VersionDescription;
+import org.graalvm.options.OptionCategory;
+import org.graalvm.polyglot.Context;
 import org.graalvm.polyglot.PolyglotException;
 import org.graalvm.polyglot.PolyglotException.StackFrame;
 import org.slf4j.LoggerFactory;
@@ -59,7 +63,7 @@ import scala.util.Right$;
 import scala.util.Success;

 /** The main CLI entry point class. */
-public final class Main {
+public final class Main extends org.graalvm.launcher.AbstractLanguageLauncher {
   private static final String RUN_OPTION = "run";
   private static final String INSPECT_OPTION = "inspect";
   private static final String DUMP_GRAPHS_OPTION = "dump-graphs";
@@ -1344,20 +1348,10 @@ public final class Main {
     }
   }

-  private void println(String msg) {
-    out().println(msg);
-  }
-
-  private void launch(String[] args) {
-    var options = buildOptions();
-    var line = preprocessArguments(options, args);
-    launch(options, line);
-  }
-
-  protected CommandLine preprocessArguments(Options options, String[] args) {
+  private CommandLine preprocessCommandLine(Options options, String[] args) {
     var parser = new DefaultParser();
     try {
-      var line = parser.parse(options, args);
+      line = parser.parse(options, args);
       return line;
     } catch (Exception e) {
       printHelp(options);
@@ -1365,7 +1359,7 @@ public final class Main {
     }
   }

-  protected void launch(Options options, CommandLine line) {
+  private void launch(Options options, CommandLine line) {
     var logLevel =
         scala.Option.apply(line.getOptionValue(LOG_LEVEL))
             .map(this::parseLogLevel)
@@ -1401,10 +1395,35 @@ public final class Main {
     }
   }

+  @Override
   protected String getLanguageId() {
     return LanguageInfo.ID;
   }

+  private final Options options = buildOptions();
+  private CommandLine line;
+
+  @Override
+  protected List<String> preprocessArguments(
+      List<String> arguments, Map<String, String> polyglotOptions) {
+    if (arguments.stream().filter(a -> a.startsWith("--help")).findFirst().isPresent()) {
+      return arguments;
+    }
+    var args = arguments.toArray(new String[0]);
+    line = preprocessCommandLine(options, args);
+    return Collections.emptyList();
+  }
+
+  @Override
+  protected void launch(Context.Builder contextBuilder) {
+    launch(options, line);
+  }
+
+  @Override
+  protected void printHelp(OptionCategory maxCategory) {
+    printHelp(options);
+  }
+
   private static final class WrongOption extends Exception {
     WrongOption(String msg) {
       super(msg);
diff --git engine/runtime-fat-jar/src/main/java/module-info.java engine/runtime-fat-jar/src/main/java/module-info.java
index ca8f410fa4..d1800ab6fc 100644
--- engine/runtime-fat-jar/src/main/java/module-info.java
+++ engine/runtime-fat-jar/src/main/java/module-info.java
@@ -5,6 +5,7 @@ open module org.enso.runtime {
   requires java.se;
   // Because of akka.util.Unsafe
   requires jdk.unsupported;
+  requires org.graalvm.launcher;
   requires org.graalvm.polyglot;
   requires org.graalvm.truffle;
   requires static org.slf4j;
diff --git project/GraalVM.scala project/GraalVM.scala
index 06431bf264..17177e13a5 100644
--- project/GraalVM.scala
+++ project/GraalVM.scala
@@ -16,6 +16,8 @@ object GraalVM {
     * When invoking the `java` command, these modules need to be put on the module-path.
     */
   val modules: Seq[ModuleID] = Seq(
+    "org.graalvm.sdk"      % "launcher-common"  % version,
+    "org.graalvm.shadowed" % "jline"            % version,
     "org.graalvm.sdk"      % "nativeimage"      % version,
     "org.graalvm.sdk"      % "word"             % version,
     "org.graalvm.sdk"      % "jniutils"         % version,

one gets basic integration with Truffle launcher. CCing @Akirathan

JaroslavTulach commented 2 months ago

With the above change I am getting:

/bin/enso --help
usage: enso
    --auth-token <token>                   Authentication token for the
                                           upload.
    --compile <package>                    Compile the provided package
                                           without executing it.
    --daemon                               Daemonize Language Server
    --data-port <data-port>                Data port for visualization
                                           protocol
    --disable-private-check                Disables private module
                                           checking at runtime. Useful for
                                           tests.
    --docs                                 Runs the Enso documentation
                                           generator.
    --dump-graphs                          Dumps IGV graphs when --run is
                                           used.
    --execution-environment <name>         Execution environment to use
                                           during execution
                                           (`live`/`design`). Defaults to
                                           `live`.
 -h,--help                                 Displays this message.
    --hide-progress                        If specified, progress bars
                                           will not be displayed.
    --in-project <project-path>            Setting this option when
                                           running the REPL or an Enso
                                           script, runs itin context of
                                           the specified project.
    --inspect                              Start the Chrome inspector when
                                           --run is used.
    --interface <interface>                Interface for processing all
                                           incoming connections
    --ir-caches                            Enables IR caches. These are on
                                           by default in production builds
                                           and off by default in developer
                                           builds. You may not specify
                                           this option with
                                           `--no-ir-caches`.
    --json                                 Switches the --version option
                                           to JSON output.
    --log-level <log-level>                Sets the runtime log level.
                                           Possible values are: OFF,
                                           ERROR, WARNING, INFO, DEBUG and
                                           TRACE. Default: INFO.
    --logger-connect <uri>                 Connects to a logging service
                                           server and passes all logs to
                                           it.
    --new <path>                           Creates a new Enso project.
    --new-project-author-email <email>     Specifies the email of the
                                           author and maintainer of the
                                           project created using --new.
    --new-project-author-name <name>       Specifies the name of the
                                           author and maintainer of the
                                           project created using --new.
    --new-project-name <name>              Specifies a project name when
                                           creating a project using --new.
    --new-project-normalized-name <name>   Specifies a normalized
                                           (Upper_Snake_Case) name when
                                           creating a project using --new.
    --new-project-template <name>          Specifies a project template
                                           when creating a project using
                                           --new.
    --no-compile-dependencies              Tells the compiler to not
                                           compile dependencies when
                                           performing static compilation.
    --no-global-cache                      Tells the compiler not to write
                                           compiled data to the global
                                           cache locations.
    --no-ir-caches                         Disables IR caches. These are
                                           on by default in production
                                           builds and off by default in
                                           developer builds. You may not
                                           specify this option with
                                           `--ir-caches`.
    --no-log-masking                       Disable masking of personally
                                           identifiable information in
                                           logs. Masking can be also
                                           disabled with the
                                           `NO_LOG_MASKING` environment
                                           variable.
    --no-read-ir-caches                    Disables the reading of IR
                                           caches in the runtime if IR
                                           caching is enabled.
    --path <path>                          Path to the content root.
    --preinstall-dependencies              Installs dependencies of the
                                           project.
    --profiling-path <file>                The path to the Language Server
                                           profiling file.
    --profiling-time <seconds>             The duration in seconds
                                           limiting the profiling time.
    --repl                                 Runs the Enso REPL.
    --root-id <uuid>                       Content root id.
    --rpc-port <rpc-port>                  RPC port for processing all
                                           incoming connections
    --run <file>                           Runs a specified Enso file.
    --secure-data-port <data-port>         A secure data port for
                                           visualization protocol
    --secure-rpc-port <rpc-port>           A secure RPC port for
                                           processing all incoming
                                           connections
    --server                               Runs Language Server
    --skip-graalvm-updater                 Skips GraalVM and its
                                           components setup during
                                           bootstrapping.
    --update-manifest                      Updates the library manifest
                                           with the updated list of direct
                                           dependencies.
    --upload <url>                         Uploads the library to a
                                           repository. The url defines the
                                           repository to upload to.
    --version                              Checks the version of the Enso
                                           executable.
    --warnings-limit <limit>               Specifies a maximal number of
                                           reported warnings. Defaults to
                                           `100`.
    --with-auto-parallelism                Enables auto parallelism in the
                                           Enso interpreter.

Runtime options:
  --polyglot                                   Run with all other guest languages accessible.
  --native                                     Run using the native launcher with limited access to Java libraries (default).
  --jvm                                        Run on the Java Virtual Machine with access to Java libraries.
  --vm.[option]                                Pass options to the host VM. To see available options, use '--help:vm'.
  --log.file=<String>                          Redirect guest languages logging into a given file.
  --log.[logger].level=<String>                Set language log level to OFF, SEVERE, WARNING, INFO, CONFIG, FINE, FINER, FINEST
                                               or ALL.
  --help                                       Print this help message.
  --help:vm                                    Print options for the host VM.
  --help:engine                                Print engine options.
  --help:compiler                              Print engine compiler options.
  --help:all                                   Print all options.
  --version:graalvm                            Print GraalVM version information and exit.
  --show-version:graalvm                       Print GraalVM version information and continue execution.

Languages:
  [id]                [name]                          [website]
  arrow               Truffle implementation of Arrow 
  enso                Enso                            
  js                  JavaScript                      https://www.graalvm.org/javascript/
  python              Python                          https://www.graalvm.org/python/

Tools:
  [id]                [name]                          [website]
  agentscript         Agent Script                    
  cpusampler          CPU Sampler                     https://www.graalvm.org/tools/profiling/
  cputracer           CPU Tracer                      https://www.graalvm.org/tools/profiling/
  dap                 Debug Protocol Server           https://www.graalvm.org/tools/dap/
  enso-debug-server                                   
  enso-runtime-server                                 
  heapmonitor         Heap Allocation Monitor         
  insight             Insight                         https://www.graalvm.org/tools/graalvm-insight/
  inspect             Chrome Inspector                https://www.graalvm.org/tools/chrome-debugger/
  memtracer           Memory Tracer                   https://www.graalvm.org/tools/profiling/

  Use --help:[id] for component options.

See http://www.graalvm.org for more information.
JaroslavTulach commented 1 month ago

There is a paragraph about a role of proper launcher in the #10121 discussion.

JaroslavTulach commented 1 week ago

There are two objectives of using the Truffle launcher:

Re-using the Truffle launcher would give us both - however we can probably also achieve both goals via different means and implementation.