Requiring Java compilation for generation dramatically slows API iteration in hybrid development

[stuartmorgan]

This is similar in some ways to https://github.com/dart-lang/native/issues/1660, but filing for separate discussion since that's about the Dart side, and this is about the Java side.

In my experience doing hybrid (part Dart, part native code) plugin development, it's common to need to iterate on the API boundary during the development process. Sometimes that iteration is driven by changes on the Dart side, sometimes on the native code side. Often when I find something I need to adjust, I am mid-change.

• In Pigeon, I just go change my API definition file, re-run the generated (almost instant), and continue working
• With Obj-C ffigen, I go change the relevant header(s) and re-run ffigen (very fast if I don't autoformat) and continue working
• With jnigen, I have to build the entire app. Not only is that orders of magnitude slower than the other two, it requires that all of my Java code be in a good state (and currently my Dart code due to https://github.com/dart-lang/native/issues/1660, but let's imagine that's fixed for the purposes of this discussion).

Having to stop, clean up any code I have that's in an intermediate state, and then build the entire app, just to iterate my Dart code, is a much worse experience than the Pigeon equivalent. Is there some way we could have a fast-iteration mode that's based on source without it being a config-level option, maybe?

[HosseinYousefi]

I wouldn't expect the typical JNIgen workflow to involve writing much Java code in the future.

Right now we can only implement interfaces in Dart and can't subclass for example so some Java code is necessary for that. But if all of these features get added, then you only build once, generate any number of times as the underlying library has not changed, and write everything in Dart instead of Java, so not only you get fast iteration speeds, but even hot reload on your example project which you wouldn't with Pigeon.

Is there some way we could have a fast-iteration mode that's based on source without it being a config-level option, maybe?

The source parser we currently use (doclet) is used for generating javadoc HTML from the source code but it has the limitation that it must know all imports everywhere because if you have import foo.* and import bar.*, but these are not known, the parser doesn't know if some class Baz is coming from foo or bar. We could have some heuristics, or use another parser that changes all unknown Baz to Object but that requires a lot of time to implement and it only solves the issue for Java. You'd still have the same problem if you chose to write Kotlin.

Instead please let me know where you need to write Java code and I try to prioritize features that makes it easier to transfer them all to Dart.

[HosseinYousefi]

In the meantime, if you don't use any import foo.* in your application and always fully write the import, the parser should not theoretically be confused ~but I haven't tried it myself~.

You can quickly try it adding

source_path:
  - 'android/src/main/java' # the path to you java files

to your jnigen.yaml. The source_path has the priority so your java classes will be generated from path, while the other ones keep being generated from the cached jars.

Edit: Actually in_app_java example already uses this. [Java source / jnigen.yaml / Generated Dart]

You can see that it does work because this imports some android libraries, but it never uses wildcard imports, and the generated Dart bindings have the parameter names, which means they are indeed generated from source as opposed to byte code.

[stuartmorgan]

I wouldn't expect the typical JNIgen workflow to involve writing much Java code in the future. [...] But if all of these features get added, then you only build once, generate any number of times as the underlying library has not changed, and write everything in Dart instead of Java [...] Instead please let me know where you need to write Java code and I try to prioritize features that makes it easier to transfer them all to Dart.

I think writing off hybrid development as a viable long-term model that we need to have a good support story for would be problematic; I'm happy to discuss further offline.