Add support for MethodHandles.Lookup

[remkop]

Applications that use Java 9's modules need to configure their module to allow picocli reflective access to the annotated classes and fields.

Often applications want the annotated classes and fields to be private; there should be no need to make them part of the exported API of your module just to allow picocli to access them. The below settings make this possible.

Example module-info.java:

module com.yourorg.yourapp {
    requires info.picocli;

    // Open this package for reflection to external frameworks.
    opens your.package.using.picocli;

    // or: limit access to picocli only
    opens other.package.using.picocli to info.picocli;
}

Note that neither package is exported, so other modules cannot accidentally compile against types in these packages.

Alternatively:

// open all packages in the module to reflective access
open module com.yourorg.yourapp {
    requires info.picocli;
}

[kaspernielsen]

Support for MethodHandles.Lookup would be really nice as well, a bit more flexible then fiddling with module-info.java.

However, a bit hard I guess with the requirement on Java 1.5

[remkop]

@kaspernielsen Can you explain a bit more about what you’re looking to do and how picocli support for MethodHandles.Lookup would help?

[kaspernielsen]

@remkop Here is a good overview of MethodHandles.Lookup vs explicitly opening the packages. http://in.relation.to/2017/04/11/accessing-private-state-of-java-9-modules/

Basically, I would be able to just passe a MethodHandles.Lookup to Picocli instead of having to modify module-info.java.

[remkop]

@kaspernielsen Thanks for the link, that explained it. MethodHandles.Lookup is more fine-grained than opening access in module-info.java, but that also means application authors would need to do more work; if my understanding is correct, the application would need to pass in a Lookup for every annotated field and method that picocli might reflectively access. I need to think about this a bit...

Recently I've been thinking of ways to avoid reflective access altogether, partly to facilitate running on Graal. It looks like GraalVm does not support MethodHandles (I could be wrong about this).

I've started working on an annotation processor to build a CommandSpec model from a class with picocli annotations at compile time rather than at runtime. I'm hoping to evolve this into an annotation processor that generates accessors for the annotated fields, that would allow picocli to inject values into these fields by simply calling these accessors without using reflection. This work has only just begun and is at a very eary stage, but this may be a better way going forward.

[kaspernielsen]

@remkop No, you would only need to pass it once per module. It captures the permissions of the module that creates the MethodHandles.Lookup object.

Right now GraalVM does not support MethodHandles. There are two types the Direct ones which are basically like normal method, you can actually wrap and unwrap to normal methods. I'm pretty sure they will be supported once when they release GraalVM for Java 11. Non-direct ones where you can build "code" at runtime will most likely not be supported.

Sounds cool with annotation processor, hope it will still be possible to use to do it runtime as well.

and

[kaspernielsen]

@remkop Just to follow up on this. If you want to access fields on classes that are located in named modules. You basically have 2 choices going forward, module-info.java or MethodHandles.Lookup. I can't see how you would be able to generate accessors without using one way of the two. MethodHandles.Lookup definitely being the most flexible.

I also think you will have a hard time keeping both <Java 9 and >=Java 9 in the same codebase. At least if you want it as compact as now.

[remkop]

For now, the picocli jar is an automatic module, so modular applications can use it with a module-info.java like this:

module com.yourorg.yourapp {
    requires info.picocli;
    opens package.using.picocli to info.picocli;
}

This does the job and does not introduce version issues for picocli.

I may add support for MethodHandles in the future, at which point we need to find some solution for the Java version issue.

[remkop]

Thanks for educating me on MethodHandles, it does seem like this is the future-proof way to do reflection.

[remkop]

This may not be as difficult as I thought. Picocli already has an abstraction mechanism for getting and setting values from annotated fields and methods: the CommandLine.Model.IGetter and CommandLine.Model.ISetter interfaces.

Providing a MethodHandles.Lookup-based implementation of these interfaces should not be too difficult. The problem is wiring things up, since currently during the construction of a CommandLine object, picocli will initialize these getters and setters with reflection-based implementations. If the client application's module-info does not open the application to picocli, currently an InaccessibleObjectException is thrown during the construction of a CommandLine object.

One idea for addressing this would be to introduce a new interface IPrivateAccessStrategy. There would need to be additional CommandLine(Object, IFactory, IPrivateAccessStrategy) constructors and CommandSpec.forAnnotatedObject(Object, IFactory, IPrivateAccessStrategy) factory methods to allow clients to specify an instance.

The existing CommandLine constructors would use a default IPrivateAccessStrategy implementation that creates the current reflection-based field/method getters and setters.

A custom IPrivateAccessStrategy implementation could create MethodHandles.Lookup-based getters and setters. This implementation could live in the picocli-jpms-module subproject. Java 9-based client applications could use code like below to construct a CommandLine object:

import picocli.CommandLine.Model.IPrivateAccessStrategy;
import picocli.module.LookupAccessStrategy; // in picocli-jpms-module

@Command
class MyApp {
    public static void main(String... args) {
        MethodHandles.Lookup lookup = MethodHandles.lookup(); // client code creates the Lookup
        IPrivateAccessStrategy privateAccess = new LookupAccessStrategy(lookup);
        CommandLine cmd = new CommandLine(new MyApp(), privateAccess);
        cmd.execute(args);
    }
    // ...
}

The contract of this interface could look something like this:

interface IPrivateAccessStrategy {
    IGetter createFieldValueGetter(Field, IScope, CommandSpec);
    ISetter createFieldValueSetter(Field, IScope, CommandSpec);
    IGetter createMethodValueGetter(Method, IScope, CommandSpec);
    ISetter createMethodValueSetter(Method, IScope, CommandSpec);
}

[kaspernielsen]

That sounds very promising, I hope I will have some time to look into it this week. Thanks