Multiple commands without a common parent command

[sebastienvermeille]

Hi, I'm facing some troubles with my migration from commons-cli to picocli.

Currently the project I migrate provide such structures for CLI calls: (adapted for the example)

// create a new user
java -jar my.jar -create-user -username John -password s3cr3t (+ database parameters aka -dbhost, -dbname, -dbpassword etc <--- this could fit into a picocli mixin if I am right)

// get list of all users
java -jar my.jar -users (+ database parameters)

// reset administrator user password
java -jar my.jar -reset-admin (prompt "new password") (+ database parameters)
// or
java -jar my.jar -reset-admin <new password> -y <-- force (+ database parameters)

We can just imagine it like that it's enough for the example.

Based on that I don't have any "root/main command" like in the examples: "git fetch, git clone etc. I just have - fetch, clone etc.

How could we achieve such behavior with picocli ?

I don't want to have multiple main classes + a bash script to run them as suggested here: https://github.com/remkop/picocli/issues/518#issuecomment-437382959

I want the args[] validation fully managed at java side. Not shared between shell scripts and java.

The only shell script I would like is one which runs: java -jar my.jar args$ in order to obtain ./myapp but it's something else. Out of scope for picocli imho.

Any help on it ? Thank you in advance and great job for picocli it looks really promising :+1:

[remkop]

Packaging Commands

Question: imagine you released your application as a native executable. Would you prefer to create separate executables for each command, or a single executable with subcommands?

This is how I like to think about things when considering how to package my commands. The fact that we need to invoke a command that was written in java by specifying java -jar my.jar or java -cp myclasspath my.package.MyCommand sometimes obfuscates things a bit.

Option 1. Separate Executables

If your commands were separate native executables, users would run them like this:

# as separate executables
create-user -username John -password s3cr3t ...
users (+ database parameters)
reset-admin <new password> -y ...

Now, if you want to go with option 1, and release multiple top-level commands, you have several ways to accomplish this:

• create multiple jars, each with its own "main-class" in the manifest
• have a single jar, and ask your users to invoke the commands with java -cp myclasspath my.package.MyCommand
• have a single jar, and create multiple wrapper scripts so your users can type ./create-user instead of java -cp myclasspath my.package.CreateUserCommand
• compile each command to a separate GraalVM native image

Option 2. Single Executable with Subcommands

Or would you prefer to have a single executable, and users provide command line options to indicate which functionality in the app they want to invoke? That would look something like this:

# as a single executable ("manage-users", for example)
manage-users create-user -username John -password s3cr3t ...
manage-users list-users (+ database parameters)
manage-users reset-admin <new password> -y ...

If you want to go with option 2, and release a single top-level command, then it seems like a natural choice to model things like create-user, list-users and reset-admin as subcommands. Modelling them as subcommands has the advantage that users can get a nice overview of the available commands when they type manage-users --help. So that would be my first intuition.

My Recommendation

I would recommend option 2.

From your example, I am guessing that in the commons-CLI-based application you are migrating from, these are modelled as options instead of subcommands. If you want to retain backwards compatibility, you could consider naming the subcommands with a leading - hyphen, or even better, give your subcommands an alias with a leading - hyphen:

@Command(name = "create-user", aliases = "-create-user", ...

Anyway, a single top-level command with subcommands seems to me to be the most natural way to model your application. It would also mean that you can publish a single jar, and users can run the application (the top-level command) by specifying java -jar my.jar. (As you pointed out, you can still have a single wrapper script if you desire.)

[sebastienvermeille]

Hi, thank you for your answer :)

Correct me if I am wrong but if I understood correctly:

Suppose I have 20 commands, I should: a) create 20 maven modules (one per command) b) each of these module contains just a main class so that picocli is able to handle the args[] the way I want c) make each module require a common "api module" and make the main method invoke the right api service and execute right things in the db.

Basically it means I will have 20 maven modules with just one single class (with a main method) inside of each of them + generate 20 different jars.

I don't say it won't work but... only using java it means usage is crappy:

java -jar actionOne.jar -x-y-y-z-d-a
java -jar actionTwo.jar -x-y-z--sa
java -jar actionThree.jar -x-y-a-a-
...

So now assuming I keep this way of doing. From an UX perspective I need something pretty in front of that because it's really not pretty at all to have to run such commands.

GraalVM could help at it ? can I configure it in order to get something like that in the end ?

./mytool create-user -username john -password s3cret  -dbhost localhost ...
./mytool create-demo-data -dbhost localhost ...

?

If yes then it becomes okay to me but I would need more details about it.

Does it run on all linux machines without recompiling it ? Will that work inside of a docker container without having to install any extra package ?

Thank you very much for your answers I hope we can solve this issue.

[remkop]

Hi, I realized that my answer wasn't very clear so I rewrote it.

I would not recommend creating separate jars for each command. I would recommend that you model your application to have a single top-level command and make create-user, create-demo-data, etc. subcommands of that top-level command. You can then have a single jar.

Users can run your jar like this:

java -jar myapp.jar create-user -username john -password s3cret -dbhost localhost ...
java -jar myapp.jar create-demo-date -dbhost localhost...

One you have that jar, then if you create a native image with GraalVM, users would execute it like this (just like you are looking for):

./mytool create-user -username john -password s3cret -dbhost localhost ...
./mytool create-demo-date -dbhost localhost...

With or without Graal, the first step is to create a top-level command and make create-user and the other commands subcommands of that top-level command.

[sebastienvermeille]

@remkop ahhh!!! really happy to see that it's possible like that ! I was a bit surprised of having to build multiple jars :)

So basically I have to create a single top-level command which get no args with some sub commands.

I will give it a try thank you

[remkop]

@sebastienvermeille Glad to hear that! Feel free to reopen or create a separate ticket if there’s any problem.

[fionik]

I am reading this conversation and cannot get the idea. You wrote:

Users can run your jar like this: java -jar myapp.jar create-user -username john -password s3cret -dbhost localhost ... java -jar myapp.jar create-demo-date -dbhost localhost

Does it mean that even though we will never mention a top command in the command line we will need to have a single "invisible" top command that will always be implicitly used regardless of what we write as a parameter after myapp.jar - create-user or create-demo-date?

What is the point of ever having a top command that does not mean anything, does not take any parameters? Is it just a container?

I am just trying to a apply the logic from C# CommandLineParser where we declare a number of the classes where each class represents a single command with parameters. When we parse the command line we provide essentially a list of classes that contain possible commands and in that case no any kind of top command is involved.

[remkop]

I guess the confusion is between stand-alone commands and command suites (commands with subcommands). I should take care to use the word "top-level" commands only in the context of command suites with subcommands, and otherwise use the word "stand-alone" commands.

Stand-alone Commands

Every Java class that has a main method can be a stand-alone command. The checksum example in the user manual shows how to create a standalone command with picocli.

If we want create-user or create-demo-date to be independent stand-alone commands, we just create a CreateUser class with a main method and a separate CreateDemoDate class, also with a main method. End users would run these commands like this:

java -cp lib/*.jar org.myorg.CreateUser --username ...

and

java -cp lib/*.jar org.myorg.CreateDemoDate --dbhost ...

We could compile these to native executables with GraalVM, and we would end up with two separate executables, create-user.exe and create-demo-date.exe. End users would then be able to invoke our commands with a more "natural" name:

create-user --username ...

and

create-demo-date --dbhost ...

Command Suites with Subcommands

If we have many commands, we could consider creating a command suite with subcommands. The entry point of a command suite is the top-level command. This top-level command must have a main method. Subcommands do not need a main method.

Suppose we create a command suite "myapp" and we create a class org.myorg.MyApp with a main method as the top-level command. We make create-user and create-demo-date subcommands of myapp.

We could compile our command suite to a single native executable with GraalVM, called myapp.exe. End users would then invoke the myapp top-level command and specify the subcommand as the first parameter:

myapp create-user --username ...

and

myapp create-demo-date --dbhost ...

The equivalent way to run these commands when using normal Java looks like this:

java -cp lib/*.jar org.myorg.MyApp create-user --username ...

and

java -cp lib/*.jar org.myorg.MyApp create-demo-date --dbhost ...

Note that with a command suite, we have a single main class, so we could make our jar an "executable jar" by putting Main-Class: org.myorg.MyApp in the META-INF/MANIFEST.MF in our jar. That would allow end users to run our commands with the -jar option and omit the org.myorg.MyApp main class name. For example:

java -jar myapp.jar create-user --username ...

and

java -jar myapp.jar create-demo-date --dbhost ...

Creating an executable jar makes sense for command suites because they have a single top-level command, but it makes less sense when you have multiple stand-alone commands.

Did this answer your question?

[Curtisjk]

@remkop - I'm trying to apply the same pattern too (option 2 - single executable with multiple subcommands) - I've got a basic skeleton parent command with 2 subcommands working, but I was wondering if there was any way to force the usage when no subcommand is presented?

At present I have the following:

@Command(subcommands = {Utility.Checker.class, Utility.Clearer.class})
public class Utility implements Runnable {

    @Override
    public void run() {
        System.out.println("In Utility");
    }

    @Command(name = "check")
    static class Checker implements Runnable {

        @Override
        public void run() {
            System.out.println("In Checker");
        }
    }

    @Command(name = "clear-all")
    static class Clearer implements Runnable {

        @Override
        public void run() {
            System.out.println("In Clearer");
        }
    }

    public static void main(String... args) {
        int exitCode = new CommandLine(new Utility()).execute(args);
        System.exit(exitCode);
    }
}

java -jar target/cmd-utility-4.63.0.0-SNAPSHOT.jar
In Utility
java -jar target/cmd-utility-4.63.0.0-SNAPSHOT.jar check
In Checker
java -jar target/cmd-utility-4.63.0.0-SNAPSHOT.jar clear-all
In Clearer

Ideally I do not want Utility.run() to be executed when executing the jar without any parameters, but instead some usage to be printed about which subcommands are available. Is this something picocli can achieve?

[remkop]

Hi @Curtisjk, yes this is possible. Please copy the logic from this example in the manual.

[Curtisjk]

Hi @Curtisjk, yes this is possible. Please copy the logic from this example in the manual.

That's great! May I suggest this use case (single executable with subcommands) is given as an example?

[remkop]

@Curtisjk I am actually considering making this the default behaviour if the parent command does not implement Runnable or Callable in a future release: https://github.com/remkop/picocli/issues/959

[fionik]

@remkop Yes, thanks. I understand now. To use program in a traditional way we'd use sub-commands. The concept of the top-command was rather confusing as I didn't really understand what was its purpose. It would be really easier to understand it the sub-commands were simply called "commands" and the top command was called "command collection", "command list", "command container", i.e. anything that does not states that it is a command as such.

[remkop]

I see, yes. I have heard the term "command suite" used. An example is git. The top-level git command by itself does not do anything.

[knotenpunkt]

question, if i use this technique having an empty parent-class and then only subcommands, then i get that functionality i want to have, but a wrong usage-print out.

In the usage-print there is written also the name of the empty parent command, that i dont want to have printed out, how i can hide this?

java -jar myjar.jar add adfsdfg
->
Unmatched argument at index 1: 'adfsdfg'
Usage:  NAME-OF-PARENT add

but name-of-parent should not be there

[remkop]

One idea is to give the parent command the name "" (empty string).

But ultimately what the best name is really depends on how you package and distribute your application. If you distribute your application as a single uberjar containing all dependencies, then you may want to give the parent command the name "java -jar myjar.jar": that way the usage help tells your users exactly how they should invoke the command.

See Running the Application and Packaging Your Application in the user manual for more details.