Make embedding Dart easier

[gaaclarke]

Dart Embedding Improvement Recommendations

Yesterday I tried to get Dart embedded, here are some suggestions that could make Dart easier to embed. Some of this stuff might already exist but I didn't find it.

1) Revise dart_api.h

I found places where the docstrings were out of date. Parameters had the wrong names and some were missing.

2) Docstring checks as part of CI

I recommend integrating Doxygen into the CI process since it can be setup to error out if there is a discrepancy between parameters and the docstrings.

3) Embedding example

We should have a minimal "hello world" example of embedding Dart. It should be compiled and executed as part of CI. It will serve as documentation and a simple integration test.

4) Generate platform-specific AOT bundles via compile

Right now in order to create a shared library on macOS that can be loaded by Dart you need to use a combination of gen_kernel.dart.snapshot and gen_snapshot. There are some flows that can be executed with just dart compile, like creating an elf bundle. Here's some suggestions:

• Eliminate the need to use gen_kernel.dart.snapshot, it's hard to find.
• Generate Mach bundles when executing dart compile aot on macOS.
• Eliminate gen_snapshot and move everything under dart compile.

5) Simplify API

• Eliminate the need to call Dart_SetVMFlags, default to nothing.
• Add initializers to structs.
• Add Dart_Load that takes a bundle and initializes a VM and spawns an IsolateGroup.

int main() {
  // Remove the need to call Dart_SetVMFlags, default to no flags.
  // Dart_SetVMFlags(0, NULL);

  Dart_InitializeParams params;
  // Just sets up pass-through callbacks and file handling with the c standard
  // library.
  Dart_InitializeParamsSetDefaults(&params);

  Dart_IsolateFlags flags;
  Dart_IsolateFlagsSetDefaults(&flags);

  // Dart_Load will initialize a VM with a bundle (.DLL, .dylib, or .so)
  // depending on the platform (we will need an API or environment variable to set
  // where the bundles are found).  It will also create an isolate group and spawn
  // the entry point specified in he isolate flags.
  Dart_Isolate isolate =
      Dart_Load(/*bundle_name=*/"hello",
                /*initialize_params=*/&params,
                /*isolate_flags=*/&flags);
}

6) Library for auto marshaling

As a quality of life improvement, with C++'s type system we could automatically marshal input parameters and results so clients don't have to do that manually. The same thing could be accomplished with C by implementing our own name mangling scheme like JNI does.

int main() {
  Dart_Isolate isolate = StartIsolate();
  Dart_Handle klass = Dart_GetClass("foo", "Adder");
  int result1 = dart::invoke<int>(klass, "Add", 1, 2);
  int result2 = Dart_InvokeNNN(klass, "Add", 1, 2);
  Dart_KillIsolate(isolate);
}

cc @mit-mit

[zanderso]

cc @a-siva @mraleph

[mraleph]

At some point I have started creating embedding examples here (they currently cover only JIT, not AOT though).

However it is not entirely clear to me if we necessarily want to make Dart VM more embeddable than it already is. I think people should prefer to embed their native code into Dart VM rather than Dart VM into their native code.

I would be curious to hear a bit more about your use cases @gaaclarke

[gaaclarke]

My personal use case this time was creating a testbed to test Dart features for Flutter outside of Flutter so that I could get a better understanding the Dart API.

I've embedded other languages before in other apps, Lua, C#, Gambit, etc. I've done it to create game engines where the renderer is optimized c++ and calls into the hosted language for game logic. I've done it to create programming sandboxes for educational software, host user written plugins, and I've temporarily put them in place before to get the benefits of not having to recompile while developing logic, then ripped them with more optimized code.

As Dart becomes more popular we'll want people to build up its mindshare by helping them embed it. The language has always been about embedding. It has some benefits like hot-reloading + AOT that would make it appealing. The current state of our support could be a turn off to someone evaluating embedded VMs, despite us having some technical advantages.