= CREST
Command-line API styled after JAX-RS
CREST allows you to get to the real work as quickly as possible when writing command line tools in Java.
Simply annotate the parameters of any Java method so it can be invoked from a command-line interface with near-zero additional work. Command-registration, help text and validation is taken care of for you.
== Start your project
Use the Maven archetype and run your first command now. Copy the following commands and paste them into your terminal.
If all went well you should see the following output:
Hello, World!
Yes, you can actually create executable command-line programs in Java!
== Example: rsync as a Crest command
For example, to do something that might be similar to rsync in java, you could create the following method signature in any java object.
package org.example.toolz;
import org.tomitribe.crest.api.Command; import org.tomitribe.crest.api.Default; import org.tomitribe.crest.api.Option;
import java.io.File; import java.net.URI; import java.util.regex.Pattern;
public class AnyName {
@Command
public void rsync(@Option("recursive") boolean recursive,
@Option("links") boolean links,
@Option("perms") boolean perms,
@Option("owner") boolean owner,
@Option("group") boolean group,
@Option("devices") boolean devices,
@Option("specials") boolean specials,
@Option("times") boolean times,
@Option("exclude") Pattern exclude,
@Option("exclude-from") File excludeFrom,
@Option("include") Pattern include,
@Option("include-from") File includeFrom,
@Option("progress") @Default("true") boolean progress,
URI[] sources,
URI dest) {
// TODO write the implementation...
}
Some quick notes on @Command
usage:
@Command
are allowed@Command
methods are allowed in a class@Command
methods in a class may have the same or different name@Command
=== Executing the Command
Pack this class in an uber jar with the Crest library and you could execute this command from the command line as follows:
$ java -jar target/toolz-1.0.0-SNAPSHOT.jar rsync Missing argument: URI...
Usage: rsync [options] URI... URI
Of course, if we execute the command without the required arguments it will error out. This is the value of Crest -- it does this dance for you.
In a dozen and more years of writing tools on different teams, two truths seem to prevail:
Computers are easy, humans are complex. Let Crest deal with the humans, you just write code.
== Help Text
In the above example we have no details in our help other than what can be generated from inspecting the code. To add actual descriptions to our
code we simply need to put an OptionDescriptions.properties
in the same package as our class.
Some quick notes on OptionDescription.properties
files:
java.util.ResourceBundle
objects, so i18n is supportedOptionDescription_en.properties
and similar for Locale specific help text@Command
in the package shares the same OptionDescription
ResourceBundle and keys<command>.<option>
as the key for situations where sharing is not desiredWith the above in our classpath, our command's help will now look like the following:
$ java -jar target/toolz-1.0.0-SNAPSHOT.jar rsync Missing argument: URI...
Usage: rsync [options] URI... URI
== Bash Completion
For those that use Bash as their shell, command completion can be added to your environment as follows:
Using the example "toolz" cli created above, this could work like so:
If the toolz-1.0.0-SNAPSHOT.jar was turned into an executable file called toolz
via the really-executable-jar
maven plugin and toolz
is in the system PATH
, this would also work:
== @Default values
Setting defaults to the @Option
parameters of our @Command
method can be done via the @Default
annotation. Using as
simplified version of our rsync
example, we might possibly wish to specify a default exclude
pattern.
@Command public void rsync(@Option("exclude") @Default(".*~") Pattern exclude, @Option("include") Pattern include, @Option("progress") @Default("true") boolean progress, URI[] sources, URI dest) {
// TODO write the implementation...
Some quick notes about @Option
:
@Option
parameters are, by default, optional@Default
is not used, the value will be its equivalent JVM default -- typically 0
or null
@Required
to force a user to specify a valueDefault values will show up in help output automatically, no need to update your OptionDescriptions.properties
Usage: rsync [options] URI... URI
=== Advanced
Default values also support interpolations:
env
is a prefix used to read the default in the environment variables and sys
to read the system properties.
TIP: you can also register custom DefaultsContext
in the interpolation registry using META-INF/services/org.tomitribe.crest.contexts.DefaultsContext
file to register it (just put a fully qualified implementation per line). The prefix will be the simple name of the implementation in lowercase. For instance
org.company.MyEnv
will use myenv
.
Finally the interpolation in such a form supports defaults:
== @Option Lists and Arrays
There are situations where you might want to allow the same flag to be specified twice. Simply turn the @Option
parameter into an
array or list that uses generics.
@Command public void rsync(@Option("exclude") @Default(".*~") Pattern[] excludes, @Option("include") Pattern include, @Option("progress") @Default("true") boolean progress, URI[] sources, URI dest) {
// TODO write the implementation...
The user can now specify multiple values when invoking the command by repeating the flag.
== @Default @Option Lists and Arrays
Should you want to specify these two exclude
values as the defaults, simply use a comma ,
to separate them in @Default
@Command public void rsync(@Option("exclude") @Default(".\.iml,.\.iml") Pattern[] excludes, @Option("include") Pattern include, @Option("progress") @Default("true") boolean progress, URI[] sources, URI dest) {
If you happen to need comma for something, use tab \t
instead. When a tab is present in the @Default
string, it becomes the preferred splitter.
@Command public void rsync(@Option("exclude") @Default(".\.iml\t.\.iml") Pattern[] excludes, @Option("include") Pattern include, @Option("progress") @Default("true") boolean progress, URI[] sources, URI dest) {
If you happen to need both tab and comma for something (really????), use unicode zero \u0000
instead.
@Command public void rsync(@Option("exclude") @Default(".\.iml\u0000.\.iml") Pattern[] excludes, @Option("include") Pattern include, @Option("progress") @Default("true") boolean progress, URI[] sources, URI dest) {
== @Default and ${variable} Substitution
In the event you want to make defaults contextual, you can use ${some.property}
in the @Default
string and
the java.lang.System.getProperties()
object to supply the value.
== Return Values
In the above we wrote to the console, which is fine for simple things but can make testing hard. So far our commands are still POJOs and
nothing is stopping us from unit testing them as plain java objects -- except asserting output writen to System.out
.
Simply return java.lang.String
and it will be written to System.out
for you.
In the event you need to write a significant amount of data, you can return org.tomitribe.crest.api.StreamingOutput
which is an exact copy of the
equivalent JAX-RS http://docs.oracle.com/javaee/6/api/javax/ws/rs/core/StreamingOutput.html[StreamingOutput] interface.
@Command public StreamingOutput cat(final File file) { if (!file.exists()) throw new IllegalStateException("File does not exist: " + file.getAbsolutePath()); if (!file.canRead()) throw new IllegalStateException("Not readable: " + file.getAbsolutePath()); if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());
return new StreamingOutput() {
@Override
public void write(OutputStream output) throws IOException {
final InputStream input = new BufferedInputStream(new FileInputStream(file));
try {
final byte[] buffer = new byte[1024];
int length;
while ((length = input.read(buffer)) != -1) {
output.write(buffer, 0, length);
}
output.flush();
} finally {
if (input != null) input.close();
}
}
};
Note a null
check is not necessary for the File file
parameter as Crest will not let the value of any plain argument be unspecified. All parameters which do not use @Option
are treated as required
== Stream injections
Command are often linked to console I/O. For that reason it is important to be able to interact
with Crest in/out/error streams. They are provided by the contextual Environment
instance and using its thread local
you can retrieve them. However to make it easier to work with you can inject them as well.
Out stream (out and error ones) needs to be PrintStream
typed and input is typed as a InputStream
.
Just use these types as command parameters and decorate it with @In
/@Out
/@Err
:
NOTE: using a parameter typed Environment
you'll get it injected as well but this one is not in crest-api
.
== Custom Java Types
You may have been seeing File
and Pattern
in the above examples and wondering exactly which Java classes Crest supports parameters to @Command
methods.
The short answer is, any. Crest does not use java.beans.PropertyEditor
implementations by default like libraries such as Spring do.
After nearly 20 years of Java's existence, it's safe to say two styles dominate converting a String
into a Java object:
java.io.File(String)
java.lang.Integer(String)
** java.net.URL(String)
java.util.regex.Pattern.compile(String)
java.net.URI.create(String)
** java.util.concurrent.TimeUnit.valueOf(String)
Use either of these conventions and Crest will have no problem instantiating your object with the user-supplied String
from the command-line args.
This should cover 95% of all cases, but in the event it does not, you can create a java.beans.PropertyEditor
and register it with the JVM.
Use your Google-fu to learn how to do that.
The order of precedence is as follows:
java.beans.PropertyEditor
== Custom Validation
If we look at our cat
command we had earlier and yank the very boiler-plate read/write stream logic, all we have left is some code validating the user input.
@Command public StreamingOutput cat(final File file) { if (!file.exists()) throw new IllegalStateException("File does not exist: " + file.getAbsolutePath()); if (!file.canRead()) throw new IllegalStateException("Not readable: " + file.getAbsolutePath()); if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());
return new StreamingOutput() {
@Override
public void write(OutputStream os) throws IOException {
IO.copy(file, os);
}
};
This validation code, too, can be yanked. Crest supports the use of http://beanvalidation.org[Bean Validation] to validate @Command
method
parameters.
@Command public StreamingOutput cat(@Exists @Readable final File file) { if (!file.isFile()) throw new IllegalStateException("Not a file: " + file.getAbsolutePath());
return new StreamingOutput() {
@Override
public void write(OutputStream os) throws IOException {
IO.copy(file, os);
}
};
Here we've eliminated two of our very tedious checks with Bean Validation annotations that Crest provides out of the box, but we still have one more to get rid of. We can eliminate that one by writing our own annotation and using the Bean Validation API to wire it all together.
Here is what an annotation to do the file.isFile()
check might look like -- let's call the annotation simply @IsFile
package org.example.toolz;
import javax.validation.ConstraintValidator; import javax.validation.ConstraintValidatorContext; import javax.validation.Payload; import java.io.File; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.Target;
import org.tomitribe.crest.val.Exists;
import static java.lang.annotation.ElementType.ANNOTATION_TYPE; import static java.lang.annotation.ElementType.FIELD; import static java.lang.annotation.ElementType.METHOD; import static java.lang.annotation.ElementType.PARAMETER; import static java.lang.annotation.RetentionPolicy.RUNTIME;
@Exists @Documented @javax.validation.Constraint(validatedBy = {IsFile.Constraint.class}) @Target({METHOD, FIELD, ANNOTATION_TYPE, PARAMETER}) @Retention(RUNTIME) public @interface IsFile { Class<?>[] groups() default {};
String message() default "{org.exampe.toolz.IsFile.message}";
Class<? extends Payload>[] payload() default {};
public static class Constraint implements ConstraintValidator<IsFile, File> {
@Override
public void initialize(IsFile constraintAnnotation) {
}
@Override
public boolean isValid(File file, ConstraintValidatorContext context) {
return file.isFile();
}
}
We can then update our code as follows to use this validation and eliminate all our boiler-plate.
@Command public StreamingOutput cat(@IsFile @Readable final File file) {
return new StreamingOutput() {
@Override
public void write(OutputStream os) throws IOException {
IO.copy(file, os);
}
};
Notice that we also removed @Exists
from the method parameter? Since we put @Exists
on the @IsFile
annotation,
the @IsFile
annotation effectively inherits the @Exists
logic.
Our @IsFile
annotation could inherit any number of annotations this way.
As the true strength of a great library of tools is the effort put into ensuring correct input, it's very wise to bite the bullet and proactively invest in creating a reusable set of validation annotations to cover your typical input types.
Pull requests are very strongly encouraged for any annotations that might be useful to others.
=== Bean Validation-less validations
You can also use the built-in crest validator style, it enables to lighten the dependencies by not requiring bean validation. To do that you must:
. define a custom validation annotation
. implementation the validation as a Consumer<ParamType>
or BiConsumer<AnnotationDefinedIn1, ParamType>
If the validation fails, the implementation just throws an exception with a meaningful error message.
Here is a trivial example to check a Path
is a directory:
@Target(PARAMETER) @Retention(RUNTIME) @Validation(CrestDirectory.Impl.class) public @interface CrestDirectory { }
TIP: the instances of the implementation are looked up by class in the Environment
and if none matches a plain new
is done calling the default constructor.
== Maven pom.xml setup
The following sample pom.xml will get you 90% of your way to fun with Crest and project that will output a small uber jar with all the required dependencies.
<?xml version="1.0"?> <project xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd" xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
== Bean Parameter Binding
If you don't want to inject in all your commands the same N parameters you can modelize them as an object. Just use standard parameters as constructor parameters of the bean:
To identify Color
as an "option aware" parameter just decorate it with @Options
:
@Options public class Color { // getters omitted for brevity private final int r; private final int g; private final int b; private final int a;
public Color(@Option("r") @Default("255") final int r,
@Option("g") @Default("255") final int g,
@Option("b") @Default("255") final int b,
@Option("a") @Default("255") final int a) {
this.r = r;
this.g = g;
this.b = b;
this.a = a;
}
=== Prefixing options
If you reuse the same parameter N times you'll probably want to prefix options. If we take previous example (Params
)
you can desire to use --background.r
and --foreground.r
(same for g, b, a).
Just use @Option
in the method parameter to do so:
NOTE: the '.' is not automatically added to allow you use to another convention like '-' or '_' ones for instance.
=== Override defaults
If you reuse the same parameter model accross command parameter you'll surely want to override some default in some cases.
For that purpose just use @Defaults
and define the mappings you want:
public class ColorfulCmd { @Command public static void exec(@Defaults({ @Defaults.DefaultMapping(name = "r", value = "0"), @Defaults.DefaultMapping(name = "g", value = "0"), @Defaults.DefaultMapping(name = "b", value = "0"), @Defaults.DefaultMapping(name = "a", value = "0") }) @Option("background.") final Color colorBg,
@Defaults({
@Defaults.DefaultMapping(name = "r", value = "255"),
@Defaults.DefaultMapping(name = "g", value = "255"),
@Defaults.DefaultMapping(name = "b", value = "255"),
@Defaults.DefaultMapping(name = "a", value = "255")
})
@Option("foreground.")
final Color colorFg) {
// ...
}
=== Interceptors
Sometimes you need to modify the command invocation or "insert" code before/after the command execution. For that purpose crest has some light interceptor support.
Defining an interceptor is as easy as defining a class with:
The constraint for an interceptor are:
@CrestInterceptor
CrestContext
NOTE: you can pass @CrestInterceptor
a value changing the key used to mark the interceptor.
To let a command use an interceptor or multiple ones just list them ordered in interceptedBy
parameter:
Crest supports 3 styles of declaring interceptors
==== Via @Command(interceptedBy)
The @Command
declaration uses the interceptedBy
attribute to name the interceptor class.
public static class Foo {
@Command(interceptedBy = GreenInterceptor.class)
public String fighters(final String arg) {
return arg;
}
The GreenInterceptor
definition is as usual
public class GreenInterceptor {
@CrestInterceptor
public Object intercept(final CrestContext crestContext) {
return crestContext.proceed();
}
==== Custom annotation containing @CrestInterceptor(FooInterceptor.class)
In this style, we define our own custom annotation @Red
that names RedInterceptor
directly
...and use it on our @Command
method as follows
public static class Foo {
@Red
@Command
public String fighters(final String arg) {
return arg;
}
The RedInterceptor
definition is as usual
public class RedInterceptor {
@CrestInterceptor
public Object intercept(final CrestContext crestContext) {
return crestContext.proceed();
}
==== Custom annotation containing @CrestInterceptor
loosely coupled to an implementation
In this style, we define our own custom annotation @Blue
, but it is not bound to a specific implementation. The @CrestInterceptor
does not mention the class.
The @Blue
is used on our @Command
method as in the previous example
public static class Foo {
@Blue
@Command
public String fighters(final String arg) {
return arg;
}
The BlueInterceptor
definition identifies itself as the implementation of @Blue
by using that annotation on its class
@Blue public class BlueInterceptor {
@CrestInterceptor
public Object intercept(final CrestContext crestContext) {
return crestContext.proceed();
}
This can be useful if you create an API jar where @Blue
might be contained, but you want to put the implementation in a different jar. Perhaps there are different implementations, each it it's own jar, and people choose the implementation they want by including the desired implementation jar in the classpath.
In this approach, however, it is necessary to ensure BlueInterceptor.class
is visible to Crest by creating a Loader
implementation such as the following
package org.example.myapp;
import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.Iterator; import java.util.List;
public class Loader implements org.tomitribe.crest.api.Loader {
@Override
public Iterator<Class<?>> iterator() {
final List<Class<?>> classes = new ArrayList<>();
classes.add(BlueInterceptor.class);
return classes.listIterator();
}
and declaring it in the jar at META-INF/services/org.tomitribe.crest.api.Loader
with the following contents:
==== Example for security
Crest provides a org.tomitribe.crest.interceptor.security.SecurityInterceptor
which
handles @RolesAllowed
using the SPI org.tomitribe.crest.interceptor.security.RoleProvider
to determine
if you can call or not the command contextually.
NOTE: RoleProvider
is taken from Environment
services. You can register it through org.tomitribe.crest.environments.SystemEnvironment
constructor
and just set it as environment on org.tomitribe.crest.environments.Environment.ENVIRONMENT_THREAD_LOCAL
.
Here a sample command using it:
== Maven Archetype
A maven archetype is available to quickly bootstrap small projects complete with the a pom like the above. Save yourself some time on copy/paste then find/replace.
== Maven Plugin
If you don't want to rely on runtime scanning to find classes but still want to avoid to list command classes or just reuse crest Main you can use Maven Plugin to find it and generate a descriptor used to load classes.
Here is how to define it in your pom:
== DeltaSpike Annotation Processor
Adding this dependency to your project:
Crest Generator can integrates with DeltaSpike to generate binding pojo. It will split @ConfigProperty
on first dot
and create one binding per prefix.
Here is an example:
public class DeltaspikeBean { @Inject @ConfigProperty(name = "app.service.base", defaultValue = "http://localhost:8080") private String base;
@Inject
@ConfigProperty(name = "app.service.retries")
private Integer retries;
It will generate the following binding:
package org.tomitribe.crest.generator.generated;
import java.util.Collections; import java.util.Map; import java.util.HashMap;
import org.apache.deltaspike.core.api.config.ConfigResolver; import org.apache.deltaspike.core.spi.config.ConfigSource; import org.tomitribe.crest.api.Default; import org.tomitribe.crest.api.Option;
import static java.util.Collections.singletonList;
public class App { private String serviceBase; private Integer serviceRetries;
public App(
@Option("service-base") @Default("http://localhost:8080") String serviceBase,
@Option("service-retries") Integer serviceRetries) {
final Map<String, String> ____properties = new HashMap<>();
this.serviceBase = serviceBase;
____properties.put("app.service.base", String.valueOf(serviceBase));
this.serviceRetries = serviceRetries;
____properties.put("app.service.retries", String.valueOf(serviceRetries));
ConfigResolver.addConfigSources(Collections.<ConfigSource>singletonList(new ConfigSource() {
@Override
public int getOrdinal() {
return 0;
}
@Override
public Map<String, String> getProperties() {
return ____properties;
}
@Override
public String getPropertyValue(final String key) {
return ____properties.get(key);
}
@Override
public String getConfigName() {
return "crest-app";
}
@Override
public boolean isScannable() {
return true;
}
})); }
public String getServiceBase() {
return serviceBase;
}
public void setServiceBase(final String serviceBase) {
this.serviceBase = serviceBase;
}
public Integer getServiceRetries() {
return serviceRetries;
}
public void setServiceRetries(final Integer serviceRetries) {
this.serviceRetries = serviceRetries;
}
Then you just need to reuse it ad a crest command parameter:
The nice thing is it will integrate with crest of course but also with DeltaSpike. It means the previous code
will also make DeltaSpike injection respecting App
configuration (--app-service-base=... --app-service-retries=3
for instance).
If you create a fatjar using TomEE embedded it means you can handle all your DeltaSpike configuration this way and you just need to write a TomEE Embedded runner to get DeltaSpike configuration wired from the command line:
import org.apache.tomee.embedded.Main;
Potential enhancement(s):
== Cli module
Cli module aims to provide a basic integration with JLine.
All starts from org.tomitribe.crest.cli.api.CrestCli
class. Current version is extensible through inheritance but already provides:
mycommand | jgrep foo
)org.tomitribe.crest.cli.api.CrestCli.cliHistoryFile
org.tomitribe.crest.cli.api.interceptor.interactive.Interactivable
can be used to mark a parameter as required but compatible with interactive mode
(ie the parameter is asked in interactive mode if missing).Sample usage:
TIP: CrestCli
also has a main(String[])
so it can be used directly as well.
NOTE: if you don't provide an exit
command one is added by default.
== GraalVM integration
Tomitribe Crest works very smoothly with GraalVM enabling you to get a native binary from your CLI.
You can do it writing manually your reflections.json
but you can also do it through maven using Apache Geronimo Arthur
plugin.
In this last case, you can set up your CLI auto-configuration with this setup:
IMPORTANT: requires Tomitribe Crest >= 0.17.
<.> Ensure to adjust the Graal and JVM base version (here Graal 21.3.0 in its Java 17 flavor),
<.> Reuse default Crest main,
<.> Enable crest extension for Arthur,
<.> Customize the command scanning, note that you can tune the includes/excludes and the values are comma separated and use a "start with" matching logic,
<.> If you are using @Editor
, you can control the scanning there too similarly to commands,
<.> This option is deprecated in recent graal versions so avoid a warning using a recent version, no direct link with crest itself,
Then just run: mvn install arthur:native-image
and you will get a target/<artifctId>.graal.bin
binary you can share and execute on the built platform.
IMPORTANT: only scanned editors (@Editor
) are handled by the extension, SPI ones (META-INF/services/org.tomitribe.crest.api.Editor
) can be used if you register them within GraalVM configuration and enable ServiceLoader
.
=== GraalVM example
package org.superbiz.crest.demo;
import org.tomitribe.crest.api.Command; import org.tomitribe.crest.api.Editor; import org.tomitribe.util.editor.AbstractConverter;
import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; import java.nio.file.Paths;
public class Cat { @Command(usage = "Cat a file.") public String cat(final Path file) throws IOException { return Files.readString(file); }
@Editor(Path.class)
public static class PathEditor extends AbstractConverter {
@Override
protected Object toObjectImpl(final String text) {
return Paths.get(text);
}
}
<?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
Once this project created, you can run mvn clean install arthur:native-image
.
This creates a ./target/demo-crest-arthur.graal.bin
binary and you can execute cat
command using: ./target/demo-crest-arthur.graal.bin cat <some file>
.
=== Use Crest Maven Plugin Scanning
it is also possible to make Arthur extension use crest-maven-plugin
scan goal (descriptor
).
Just set extension property tomitribe.crest.useInPlaceRegistrations
to true
:
This enables to use the same scanning for both tasks and therefore to have a common and unified scanning for java and native runs.