=clingon= is a command-line options parser system for Common Lisp.
A summary of the features supported by =clingon= is provided below.
Scroll to the demo section in order to see some examples of =clingon= in action.
Other Common Lisp option parser systems, which you might consider checking out.
Quick Example
Here's a really quick example of a simple CLI application, which greets people.
(in-package :cl-user) (defpackage :clingon.example.greet (:use :cl) (:import-from :clingon) (:export :main)) (in-package :clingon.example.greet)
(defun greet/options () "Returns the options for the `greet' command" (list (clingon:make-option :string :description "Person to greet" :short-name #\u :long-name "user" :initial-value "stranger" :env-vars '("USER") :key :user)))
(defun greet/handler (cmd) "Handler for the `greet' command" (let ((who (clingon:getopt cmd :user))) (format t "Hello, ~A!~%" who)))
(defun greet/command () "A command to greet someone" (clingon:make-command :name "greet" :description "greets people" :version "0.1.0" :authors '("John Doe <john.doe@example.org") :license "BSD 2-Clause" :options (greet/options) :handler #'greet/handler))
(defun main () "The main entrypoint of our CLI program" (let ((app (greet/command))) (clingon:run app)))
This small example shows a lot of details about how apps are structured with =clingon=.
You can see there's a =main= function, which will be the entrypoint for our ASDF system. Then you can find the =greet/command= function, which creates and returns a new command.
The =greet/options= functions returns the options associated with our sample command.
And we also have the =greet/handler= function, which is the function that will be invoked when users run our command-line app.
This way of organizing command, options and handlers makes it easy to re-use common options, or even handlers, and wire up any sub-commands anyway you prefer.
You can find additional examples included in the test suite for =clingon=.
You can also build and run the =clingon= demo application, which includes the =greet= command introduced in the previous section, along with other examples.
[[./images/clingon-demo.gif]]
Clone the [[https://github.com/dnaeon/clingon][clingon]] repo in your [[https://www.quicklisp.org/beta/faq.html][Quicklisp local-projects]] directory.
git clone https://github.com/dnaeon/clingon
Register it to your local Quicklisp projects.
CL-USER> (ql:register-local-projects)
** Building the Demo App
You can build the demo app using SBCL with the following command.
LISP=sbcl make demo
Build the demo app using Clozure CL:
LISP=ccl make demo
In order to build the demo app using ECL you need to follow these instructions, which are ECL-specific. See [[https://common-lisp.net/project/ecl/static/manual/System-building.html#Compiling-with-ASDF][Compiling with ASDF from the ECL manual]] for more details. First, load the =:clingon.demo= system.
(ql:quickload :clingon.demo)
And now build the binary with ECL:
(asdf:make-build :clingon.demo :type :program :move-here #P"./" :epilogue-code '(clingon.demo:main))
This will create a new executable =clingon-demo=, which you can now execute.
Optionally, you can also enable the bash completions support.
APP=clingon-demo source extras/completions.bash
In order to activate the Zsh completions, install the completions script in your =~/.zsh-completions= directory (or anywhere else you prefer) and update your =~/.zshrc= file, so that the completions are loaded.
Make sure that you have these lines in your =~/.zshrc= file.
fpath=(~/.zsh-completions $fpath) autoload -U compinit compinit
The following command will generate the Zsh completions script.
./clingon-demo zsh-completion > ~/.zsh-completions/_clingon-demo
Use the =--help= flag to see some usage information about the demo application.
./clingon-demo --help
Requirements
Installation
The =clingon= system is not yet part of Quicklisp, so for now you need to install it in your local Quicklisp projects.
Clone the repo in your [[https://www.quicklisp.org/beta/faq.html][Quicklisp local-projects]] directory.
(ql:register-local-projects)
Then load the system.
(ql:quickload :clingon)
In this section we will implement a simple CLI application, and explain at each step what and why we do the things we do.
Once you are done with it, you should have a pretty good understanding of the =clingon= system and be able to further extend the sample application on your own.
We will be developing the application interactively and in the REPL. Finally we will create an ASDF system for our CLI app, so we can build it and ship it.
The code we develop as part of this section will reside in a file named =intro.lisp=. Anything we write will be sent to the Lisp REPL, so we can compile it and get quick feedback about the things we've done so far.
You can find the complete code we'll develop in this section in the =clingon/examples/intro= directory.
** Start the REPL
Start up your REPL session and let's load the =clingon= system.
CL-USER> (ql:quickload :clingon) To load "clingon": Load 1 ASDF system: clingon ; Loading "clingon"
(:CLINGON)
** Create a new package
First, we will define a new package for our application and switch to it.
(in-package :cl-user) (defpackage :clingon.intro (:use :cl) (:import-from :clingon) (:export :main)) (in-package :clingon.intro)
We have our package, so now we can proceed to the next section and create our first command.
** Creating a new command
The first thing we'll do is to create a new command. Commands are created using the =CLINGON:MAKE-COMMAND= function.
Each command has a name, description, any options that the command accepts, any sub-commands the command knows about, etc.
The command in =clingon= is represented by the =CLINGON:COMMAND= class, which contains many other slots as well, which you can lookup.
(defun top-level/command () "Creates and returns the top-level command" (clingon:make-command :name "clingon-intro" :description "my first clingon cli app" :version "0.1.0" :license "BSD 2-Clause" :authors '("John Doe john.doe@example.com")))
This is how our simple command looks like. For now it doesn't do much, and in fact it won't execute anything, but we will fix that as we go.
What is important to note, is that we are using a convention here to make things easier to understand and organize our code base.
Functions that return new commands will be named =
This makes things easier later on, when we introduce new sub-commands, and when we need to wire things up we can refer to our commands using the established naming convention. Of course, it's up to you to decide which approach to take, so feel free to adjust the layout of the code to your personal preferences. In this guide we will use the afore mentioned approach.
Commands can be linked together in order to form a tree of commands and sub-commands. We will talk about that one in more details in the later sections of this guide.
** Adding options
Next, we will add a couple of options. Similar to the previous section we will define a new function, which simply returns a list of valid options. Defining it in the following way would make it easier to re-use these options later on, in case you have another command, which uses the exact same set of options.
=clingon= exposes a single interface for creating options via the =CLINGON:MAKE-OPTION= generic function. This unified interface will allow developers to create and ship new option kinds, and still have their users leverage a common interface for the options via the =CLINGON:MAKE-OPTION= interface.
(defun top-level/options () "Creates and returns the options for the top-level command" (list (clingon:make-option :counter :description "verbosity level" :short-name #\v :long-name "verbose" :key :verbose) (clingon:make-option :string :description "user to greet" :short-name #\u :long-name "user" :initial-value "stranger" :env-vars '("USER") :key :user)))
Let's break things down a bit and explain what we just did.
We've defined two options -- one of =:COUNTER= kind and another one, which is of =:STRING= kind. Each option specifies a short and long name, along with a description of what the option is meant for.
Another important thing we did is to specify a =:KEY= for our options. This is the key which we will later use in order to get the value associated with our option, when we use =CLINGON:GETOPT=.
And we have also defined that our =--user= option can be initialized via environment variables. We can specify multiple environment variables, if we need to, and the first one that resolves to something will be used as the initial value for the option.
If none of the environment variables are defined, the option will be initialized with the value specified by the =:INITIAL-VALUE= initarg.
Before we move to the next section of this guide we will update the definition of our =TOP-LEVEL/COMMAND= function, so that we include our options.
(defun top-level/command ()
"Creates and returns the top-level command"
(clingon:make-command
:name "clingon-intro"
...
:usage "[-v] [-u
** Defining a handler
A /handler/ in =clingon= is a function, which accepts an instance of =CLINGON:COMMAND= and is responsible for performing some work.
The single argument a handler receives will be used to inspect the values of parsed options and any free arguments that were provided on the command-line.
A command may or may not specify a handler. Some commands may be used purely as /namespaces/ for other sub-commands, and it might make no sense to have a handler for such commands. In other situations you may still want to provide a handler for the parent commands.
Let's define the handler for our /top-level/ command.
(defun top-level/handler (cmd) "The top-level handler" (let ((args (clingon:command-arguments cmd)) (user (clingon:getopt cmd :user)) (verbose (clingon:getopt cmd :verbose))) (format t "Hello, ~A!~%" user) (format t "The current verbosity level is set to ~A~%" verbose) (format t "You have provided ~A arguments~%" (length args)) (format t "Bye.~%")))
We are introducing a couple of new functions, which we haven't described before.
*** Positional ("free") arguments
In ~top-level/handler~, we are using =CLINGON:COMMAND-ARGUMENTS=, which returns the positional, or "free" arguments: the arguments that remain after the options are parsed. The remaining free arguments are available through ~CLINGON:COMMAND-ARGUMENTS~. In this handler we bind ~args~ to the free arguments we've provided to our command, when we invoke it on the command-line.
*** Option arguments
We also use the =CLINGON:GETOPT= function to lookup the values associated with our options. Remember the =:KEY= initarg we've used in =CLINGON:MAKE-OPTION= when defining our options?
We again update our =TOP-LEVEL/COMMAND= definition, this time with our handler included:
(defun top-level/command () "Creates and returns the top-level command" (clingon:make-command :name "clingon-intro" ... :handler #'top-level/handler)) ;; <- new code
At this point we are basically done with our simple application. But before we move to the point where build our binary and start playing with it on the command-line we can test things out on the REPL, just to make sure everything works as expected.
** Testing things out on the REPL
Create a new instance of our command and bind it to some variable.
INTRO> (defparameter app (top-level/command)) APP
Inspecting the returned instance would give you something like this.
Group slots by inheritance [ ] Sort slots alphabetically [X]
All Slots:
[ ] ARGS-TO-PARSE = NIL
[ ] ARGUMENTS = NIL
[ ] AUTHORS = ("John Doe john.doe@example.com")
[ ] CONTEXT = #<HASH-TABLE :TEST EQUAL :COUNT 0 {1004648433}>
[ ] DESCRIPTION = "my first clingon cli app"
[ ] EXAMPLES = NIL
[ ] HANDLER = #<FUNCTION TOP-LEVEL/HANDLER>
[ ] LICENSE = "BSD 2-Clause"
[ ] LONG-DESCRIPTION = NIL
[ ] NAME = "clingon-intro"
[ ] OPTIONS = (#
[set value] [make unbound]
You might also notice that besides the options we've defined ourselves, there are few additional options, that we haven't defined at all.
These options are automatically added by =clingon= itself for each new command and provide flags for =--help=, =--version= and =--bash-completions= for you automatically, so you don't have to deal with them manually.
Before we dive into testing out our application, first we will check that we have a correct help information for our command.
INTRO> (clingon:print-usage app t) NAME: clingon-intro - my first clingon cli app
USAGE:
clingon-intro [-v] [-u
OPTIONS:
--help display usage information and exit
--version display version and exit
-u, --user
AUTHORS: John Doe john.doe@example.com
LICENSE: BSD 2-Clause
NIL
This help information will make it easier for our users, when they need to use it. And that is automatically handled for you, so you don't have to manually maintain an up-to-date usage information, each time you introduce a new option.
Time to test out our application on the REPL. In order to test things out you can use the =CLINGON:PARSE-COMMAND-LINE= function by passing it an instance of your command, along with any arguments that need to be parsed. Let's try it out without any command-line arguments.
INTRO> (clingon:parse-command-line app nil)
The =CLINGON:PARSE-COMMAND-LINE= function will (as the name suggests) parse the given arguments against the options associated with our command. Finally it will return an instance of =CLINGON:COMMAND=.
In our simple CLI application, that would be the same instance as our =APP=, but things look differently when we have sub-commands.
When we start adding new sub-commands, the result of =CLINGON:PARSE-COMMAND-LINE= will be different based on the arguments it needs to parse. That means that if our input matches a sub-command you will receive an instance of the sub-command that matched the given arguments.
Internally the =clingon= system maintains a tree data structure, describing the relationships between commands. This allows a command to be related to some other command, and this is how the command and sub-commands support is implemented in =clingon=.
Each command in =clingon= is associated with a /context/. The /context/ or /environment/ provides the options and their values with respect to the command itself. This means that a parent command and a sub-command may have exactly the same set of options defined, but they will reside in different contexts. Depending on how you use it, sub-commands may /shadow/ a parent command option, but it also means that a sub-command can refer to an option defined in a global command.
The /context/ of a command in =clingon= is available via the =CLINGON:COMMAND-CONTEXT= accessor. We will use the context in order to lookup our options and the values associated with them.
The function that operates on command's context and retrieves values from it is called =CLINGON:GETOPT=.
Let's see what we've got for our options.
INTRO> (let ((c (clingon:parse-command-line app nil))) (clingon:getopt c :user)) "dnaeon" T
The =CLINGON:GETOPT= function returns multiple values -- first one specifies the value of the option, if it had any, the second one indicates whether or not that option has been set at all on the command-line, and the third value is the command which provided the value for the option, if set.
If you need to simply test things out and tell whether an option has been set at all you can use the =CLINGON:OPT-IS-SET-P= function instead.
Let's try it out with a different input.
INTRO> (let ((c (clingon:parse-command-line app (list "-vvv" "--user" "foo")))) (format t "Verbose is ~A~%" (clingon:getopt c :verbose)) (format t "User is ~A~%" (clingon:getopt c :user))) Verbose is 3 User is foo
Something else, which is important to mention here. The default precedence list for options is:
Play with it using different command-line arguments. If you specify invalid or unknown options =clingon= will signal a condition and provide you a few recovery options. For example, if you specify an invalid flag like this:
INTRO> (clingon:parse-command-line app (list "--invalid-flag"))
We will be dropped into the debugger and be provided with restarts we can choose from, e.g.
Unknown option --invalid-flag of kind LONG [Condition of type CLINGON.CONDITIONS:UNKNOWN-OPTION]
Restarts: 0: [DISCARD-OPTION] Discard the unknown option 1: [TREAT-AS-ARGUMENT] Treat the unknown option as a free argument 2: [SUPPLY-NEW-VALUE] Supply a new value to be parsed 3: [RETRY] Retry SLY mREPL evaluation request. 4: [ABORT] Return to sly-db level 1. 5: [RETRY] Retry SLY mREPL evaluation request. --more-- ...
This is similar to the way other Common Lisp options parsing systems behave such as [[https://github.com/sjl/adopt][adopt]] and [[https://github.com/libre-man/unix-opts][unix-opts]].
Also worth mentioning again here is that =CLINGON:PARSE-COMMAND-LINE= is meant to be used within the REPL, and not called directly by handlers.
** Adding a sub-command
Sub-commands are no different than regular commands, and in fact are created exactly the way we did it for our /top-level/ command.
(defun shout/handler (cmd) "The handler for the `shout' command" (let ((args (mapcar #'string-upcase (clingon:command-arguments cmd))) (user (clingon:getopt cmd :user))) ;; <- a global option (format t "HEY, ~A!~%" user) (format t "~A!~%" (clingon:join-list args #\Space))))
(defun shout/command () "Returns a command which SHOUTS back anything we write on the command-line" (clingon:make-command :name "shout" :description "shouts back anything you write" :usage "[options] [arguments ...]" :handler #'shout/handler))
And now, we will wire up our sub-command making it part of the /top-level/ command we have so far.
(defun top-level/command () "Creates and returns the top-level command" (clingon:make-command :name "clingon-intro" ... :sub-commands (list (shout/command)))) ;; <- new code
You should also notice here that within the =SHOUT/HANDLER= we are actually referencing an option, which is defined somewhere else. This option is actually defined on our top-level command, but thanks's to the automatic management of relationships that =clingon= provides we can now refer to global options as well.
Let's move on to the final section of this guide, where we will create a system definition for our application and build it.
** Packaging it up
One final piece which remains to be added to our code is to provide an entrypoint for our application, so let's do it now.
(defun main () (let ((app (top-level/command))) (clingon:run app)))
This is the entrypoint which will be used when we invoke our application on the command-line, which we'll set in our ASDF definition.
And here's a simple system definition for the application we've developed so far.
(defpackage :clingon-intro-system (:use :cl :asdf)) (in-package :clingon-intro-system)
(defsystem "clingon.intro" :name "clingon.intro" :long-name "clingon.intro" :description "An introduction to the clingon system" :version "0.1.0" :author "John Doe john.doe@example.org" :license "BSD 2-Clause" :depends-on (:clingon) :components ((:module "intro" :pathname #P"examples/intro/" :components ((:file "intro")))) :build-operation "program-op" :build-pathname "clingon-intro" :entry-point "clingon.intro:main")
Now we can build our application and start using it on the command-line.
sbcl --eval '(ql:quickload :clingon.intro)' \ --eval '(asdf:make :clingon.intro)' \ --eval '(quit)'
This will produce a new binary called =clingon-intro= in the directory of the =clingon.intro= system.
This approach uses the [[https://asdf.common-lisp.dev/asdf/Predefined-operations-of-ASDF.html][ASDF program-op operation]] in combination with =:entry-point= and =:build-pathname= in order to produce the resulting binary.
If you want to build your apps using [[https://www.xach.com/lisp/buildapp/][buildapp]], please check the /Buildapp/ section from this document.
** Testing it out on the command-line
Time to check things up on the command-line.
$ ./clingon-intro --help NAME: clingon-intro - my first clingon cli app
USAGE:
clingon-intro [-v] [-u
OPTIONS:
--help display usage information and exit
--version display version and exit
-u, --user
COMMANDS: shout shouts back anything you write
AUTHORS: John Doe john.doe@example.com
LICENSE: BSD 2-Clause
Let's try out our commands.
$ ./clingon-intro -vvv --user Lisper Hello, Lisper! The current verbosity level is set to 3 You have provided 0 arguments Bye.
And let's try our sub-command as well.
$ ./clingon-intro --user stranger shout why are yelling at me? HEY, stranger! WHY ARE YELLING AT ME?!
You can find the full code we've developed in this guide in the [[https://github.com/dnaeon/clingon/tree/master/examples][clingon/examples]] directory of the repo.
When a command needs to exit with a given status code you can use the =CLINGON:EXIT= function.
=clingon= by default will provide a handler for =SIGINT= signals, which when detected will cause the application to immediately exit with status code =130=.
If your commands need to provide some cleanup logic as part of their job, e.g. close out all open files, TCP session, etc., you could wrap your =clingon= command handlers in [[http://www.lispworks.com/documentation/HyperSpec/Body/s_unwind.htm][UNWIND-PROTECT]] to make sure that your cleanup tasks are always executed.
However, using [[http://www.lispworks.com/documentation/HyperSpec/Body/s_unwind.htm][UNWIND-PROTECT]] may not be appropriate in all cases, since the cleanup forms will always be executed, which may or may not be what you need.
For example if you are developing a =clingon= application, which populates a database in a transaction you would want to use [[http://www.lispworks.com/documentation/HyperSpec/Body/s_unwind.htm][UNWIND-PROTECT]], but only for releasing the database connection itself.
If the application is interrupted while it inserts or updates records, what you want to do is to rollback the transaction as well, so your database is left in a consistent state.
In those situations you would want to use the [[https://github.com/compufox/with-user-abort][WITH-USER-ABORT]] system, so that your =clingon= command can detect the =SIGINT= signal and act upon it, e.g. taking care of rolling back the transaction.
=clingon= can generate documentation for your application by using the =CLINGON:PRINT-DOCUMENTATION= generic function.
Currently the documentation generator supports only the /Markdown/ format, but other formats can be developed as separate extensions to =clingon=.
Here's how you can generate the Markdown documentation for the =clingon-demo= application from the REPL.
CL-USER> (ql:quickload :clingon.demo) CL-USER> (in-package :clingon.demo) DEMO> (with-open-file (out #P"clingon-demo.md" :direction :output) (clingon:print-documentation :markdown (top-level/command) out))
You can also create a simple command, which can be added to your =clingon= apps and have it generate the documentation for you, e.g.
(defun print-doc/command () "Returns a command which will print the app's documentation" (clingon:make-command :name "print-doc" :description "print the documentation" :usage "" :handler (lambda (cmd) ;; Print the documentation starting from the parent ;; command, so we can traverse all sub-commands in the ;; tree. (clingon:print-documentation :markdown (clingon:command-parent cmd) t))))
Above command can be wired up anywhere in your application.
Make sure to also check the =clingon-demo= app, which provides a =print-doc= sub-command, which operates on the /top-level/ command and generates the documentation for all sub-commands.
You can also find the generated documentation for the =clingon-demo= app in the =docs/= directory of the =clingon= repo.
** Generate tree representation of your commands in Dot
Using =CLINGON:PRINT-DOCUMENTATION= you can also generate the tree representation of your commands in [[https://en.wikipedia.org/wiki/DOT_(graph_description_language)][Dot]] format.
Make sure to check the =clingon.demo= system and the provided =clingon-demo= app, which provides an example command for generating the Dot representation.
The example below shows the generation of the Dot representation for the =clingon-demo= command.
clingon-demo dot digraph G { node [color=lightblue fillcolor=lightblue fontcolor=black shape=record style="filled, rounded"]; "clingon-demo" -> "greet"; "clingon-demo" -> "logging"; "logging" -> "enable"; "logging" -> "disable"; "clingon-demo" -> "math"; "clingon-demo" -> "echo"; "clingon-demo" -> "engine"; "clingon-demo" -> "print-doc"; "clingon-demo" -> "sleep"; "clingon-demo" -> "zsh-completion"; "clingon-demo" -> "dot"; }
+end_src
We can generate the resulting graph using [[https://graphviz.org/][graphviz]].
clingon-demo dot > clingon-demo.dot dot -Tpng clingon-demo.dot > clingon-demo-tree.png
+end_src
This is what the resulting tree looks like.
[[./images/clingon-demo-tree.png]]
=clingon= allows you to associate =pre= and =post= hooks with a command.
The =pre= and =post= hooks are functions which will be invoked before and after the respective command handler is executed. They are useful in cases when you need to set up or tear things down before executing the command's handler.
An example of a =pre-hook= might be to configure the logging level of your application based on the value of a global flag. A =post-hook= might be responsible for shutting down any active connections, etc.
The =pre-hook= and =post-hook= functions accept a single argument, which is an instance of =CLINGON:COMMAND=. That way the hooks can examine the command's context and lookup any flags or options.
Hooks are also hierachical in the sense that they will be executed based on the command's lineage.
Consider the following example, where we have a CLI app with three commands.
main -> foo -> bar
In above example the =bar= command is a sub-command of =foo=, which in turn is a sub-command of =main=. Also, consider that we have added pre- and post-hooks to each command.
If a user executed the following on the command-line:
$ main foo bar
Based on the above command-line =clingon= would do the following:
In above example that would be:
main (pre-hook)
foo (pre-hook)
bar (pre-hook)
bar (handler) bar (post-hook) foo (post-hook) main (post-hook)
+end_src
Associating hooks with commands is done during instantiation of a command. The following example creates a new command with a =pre-hook= and =post-hook=.
(defun foo/pre-hook (cmd) "The pre-hook for `foo' command" (declare (ignore cmd)) (format t "foo pre-hook has been invoked~&"))
(defun foo/post-hook (cmd) "The post-hook for `foo' command" (declare (ignore cmd)) (format t "foo post-hook has been invoked~&"))
(defun foo/handler (cmd) (declare (ignore cmd)) (format t "foo handler has been invoked~&"))
(defun foo/command () "Returns the `foo' command" (clingon:make-command :name "foo" :description "the foo command" :authors '("John Doe john.doe@example.org") :handler #'foo/handler :pre-hook #'foo/pre-hook :post-hook #'foo/post-hook :options nil :sub-commands nil))
If we have executed above command we would see the following output.
foo pre-hook has been invoked foo handler has been invoked foo post-hook has been invoked
The =CLINGON:BASE-ERROR= condition may be used as the base for user-defined conditions.
The =CLINGON:RUN= method will invoke =CLINGON:HANDLE-ERROR= for conditions which sub-class =CLINGON:BASE-ERROR=. The implementation of =CLINGON:HANDLE-ERROR= allows the user to customize the way errors are being reported and handled.
The following example creates a new custom condition.
(in-package :cl-user) (defpackage :my.clingon.app (:use :cl) (:import-from :clingon) (:export :my-app-error)) (in-package :my.clingon.app)
(define-condition my-app-error (clingon:base-error) ((message :initarg :message :initform (error "Must specify message") :reader my-app-error-message)) (:documentation "My custom app error condition"))
(defmethod clingon:handle-error ((err my-app-error)) (let ((message (my-app-error-message err))) (format error-output "Oops, an error occurred: ~A~%" message)))
You can now use the =MY-APP-ERROR= condition anywhere in your command handlers and signal it. When this condition is signalled =clingon= will invoke the =CLINGON:HANDLE-ERROR= generic function for your condition.
The default implementation of =CLINGON:RUN= provides error handling for the most common user-related errors, such as handling of missing arguments, invalid options/flags, catching of =SIGINT= signals, etc.
Internally =CLINGON:RUN= relies on =CLINGON:PARSE-COMMAND-LINE= for the actual parsing. In order to provide custom logic during parsing, users may provide a different implementation of either =CLINGON:RUN= and/or =CLINGON:PARSE-COMMAND-LINE= by subclassing the =CLINGON:COMMAND= class.
An alternative approach, which doesn't need a subclass of =CLINGON:COMMAND= is to provide =AROUND= methods for =CLINGON:RUN=.
For instance, the following code will treat unknown options as free arguments, while still using the default implementation of =CLINGON:RUN=.
(defmethod clingon:parse-command-line :around ((command clingon:command) arguments) "Treats unknown options as free arguments" (handler-bind ((clingon:unknown-option (lambda (c) (clingon:treat-as-argument c)))) (call-next-method)))
See [[https://github.com/dnaeon/clingon/issues/11][this issue]] for more examples and additional discussion on this topic.
The =clingon= system supports various kinds of options, each of which is meant to serve a specific purpose.
Each builtin option can be initialized via environment variables, and new mechanisms for initializing options can be developed, if needed.
Options are created via the single =CLINGON:MAKE-OPTION= interface.
The supported option kinds include:
** Counters Options
A =counter= is an option kind, which increments every time it is set on the command-line.
A good example for =counter= options is to provide a flag, which increases the verbosity level, depending on the number of times the flag was provided, similar to the way =ssh(1)= does it, e.g.
ssh -vvv user@host
Here's an example of creating a =counter= option.
(clingon:make-option :counter :short-name #\v :long-name "verbose" :description "how noisy we want to be" :key :verbose)
The default =step= for counters is set to =1=, but you can change that, if needed.
(clingon:make-option :counter :short-name #\v :long-name "verbose" :description "how noisy we want to be" :step 42 :key :verbose)
** Boolean Options
The following boolean option kinds are supported by =clingon=.
The =:boolean= kind is an option which expects an argument, which represents a boolean value.
Arguments =true= and =1= map to =T= in Lisp, anything else is considered a falsey value and maps to =NIL=.
(clingon:make-option :boolean :description "my boolean" :short-name #\b :long-name "my-boolean" :key :boolean)
This creates an option =-b, --my-boolean
The =:boolean/true= option kind creates a flag, which always returns =T=.
The =:boolean/false= option kind creates a flag, which always returns =NIL=.
The =:flag= option kind is an alias for =:boolean/true=.
** Integer Options
Here's an example of creating an option, which expects an integer argument.
(clingon:make-option :integer :description "my integer opt" :short-name #\i :long-name "int" :key :my-int :initial-value 42)
** Choice Options
=choice= options are useful when you have to limit the arguments provided on the command-line to a specific set of values.
For example:
(clingon:make-option :choice :description "log level" :short-name #\l :long-name "log-level" :key :choice :items '("info" "warn" "error" "debug"))
With this option defined, you can now set the logging level only to =info=, =warn=, =error= or =debug=, e.g.
-l, --log-level [info|warn|error|debug]
** Enum Options
Enum options are similar to the =choice= options, but instead of returning the value itself they can be mapped to something else.
For example:
(clingon:make-option :enum :description "enum option" :short-name #\e :long-name "my-enum" :key :enum :items '(("one" . 1) ("two" . 2) ("three" . 3)))
If a user specifies =--my-enum=one= on the command-line the option will be have the value =1= associated with it, when being looked up via =CLINGON:GETOPT=.
The values you associate with the enum variant, can be any object.
This is one of the options being used by the /clingon-demo/ application, which maps user input to Lisp functions, in order to perform some basic math operations.
(clingon:make-option :enum :description "operation to perform" :short-name #\o :long-name "operation" :required t :items `(("add" . ,#'+) ("sub" . ,#'-) ("mul" . ,#'*) ("div" . ,#'/)) :key :math/operation)
** List / Accumulator Options
The =:list= option kind accumulates each argument it is given on the command-line into a list.
For example:
(clingon:make-option :list :description "files to process" :short-name #\f :long-name "file" :key :files)
If you invoke an application, which uses a similar option like the one above using the following command-line arguments:
$ my-app --file foo --file bar --file baz
When you retrieve the value associated with your option, you will get a list of all the files specified on the command-line, e.g.
(clingon:getopt cmd :files) ;; => '("foo" "bar" "baz")
A similar option exists for integer values using the =:list/integer= option, e.g.
(clingon:make-option :list/integer :description "list of integers" :short-name #\l :long-name "int" :key :integers)
** Switch Options
=:SWITCH= options are a variation of =:BOOLEAN= options with an associated list of known states that can turn a switch /on/ or /off/.
Here is an example of a =:SWITCH= option.
(clingon:make-option :switch :description "my switch option" :short-name #\s :long-name "state" :key :switch)
The default states for a switch to be considered as /on/ are:
The default states considered to turn the switch /off/ are:
You can customize the list of /on/ and /off/ states by specifying them using the =:ON-STATES= and =:OFF-STATES= initargs, e.g.
(clingon:make-option :switch :description "engine switch option" :short-name #\s :long-name "state" :on-states '("start") :off-states '("stop") :key :engine)
These sample command-line arguments will turn a switch on and off.
my-app --engine=start --engine=stop
The final value of the =:engine= option will be =NIL= in the above example.
** Persistent Options
An option may be marked as /persistent/. A /persistent/ option is such an option, which will be propagated from a parent command to all sub-commands associated with it.
This is useful when you need to provide the same option across sub-commands.
The following example creates one top-level command (=demo= in the example), which has two sub-commands (=foo= and =bar= commands). The =foo= command has a single sub-command, =qux= in the following example.
The =top-level= command has a single option (=persistent-opt= in the example), which is marked as /persistent/.
(defun qux/command () "Returns the `qux' command" (clingon:make-command :name "qux" :description "the qux command" :handler (lambda (cmd) (declare (ignore cmd)) (format t "qux has been invoked"))))
(defun foo/command () "Returns the `foo' command" (clingon:make-command :name "foo" :description "the foo command" :sub-commands (list (qux/command)) :handler (lambda (cmd) (declare (ignore cmd)) (format t "foo has been invoked"))))
(defun bar/command () "Returns the `bar' command" (clingon:make-command :name "bar" :description "the bar command" :handler (lambda (cmd) (declare (ignore cmd)) (format t "bar has been invoked"))))
(defun top-level/command () "Returns the top-level command" (clingon:make-command :name "demo" :description "the demo app" :options (list (clingon:make-option :string :long-name "persistent-opt" :description "an example persistent option" :persistent t :key :persistent-opt)) :sub-commands (list (foo/command) (bar/command))))
Since the option is marked as persistent and is associated with the top-level command, it will be inherited by all sub-commands.
If the existing options provided by =clingon= are not enough for you, and you need something a bit more specific for your use case, then you can always implement a new option kind.
The following generic functions operate on options and are exported by the =clingon= system.
New option kinds should inherit from the =CLINGON:OPTION= class, which implements all of the above generic functions. If you need to customize the behaviour of your new option, you can still override the default implementations.
** CLINGON:INITIALIZE-OPTION
The =CLINGON:INITIALIZE-OPTION= as the name suggests is being used to initialize an option.
The default implementation of this generic function supports initialization from environment variables, but implementors can choose to support other initialization methods, e.g. be able to initialize an option from a key/value store like /Redis/, /Consul/ or /etcd/ for example.
** CLINGON:FINALIZE-OPTION
The =CLINGON:FINALIZE-OPTION= generic function is called after all command-line arguments have been processed and values for them have been derived already.
=CLINGON:FINALIZE-OPTION= is meant to /finalize/ the option's value, e.g. transform it to another object, if needed.
For example the =:BOOLEAN= option kind transforms user-provided input like =true=, =false=, =1= and =0= into their respective Lisp counterparts like =T= and =NIL=.
Another example where you might want to customize the behaviour of =CLINGON:FINALIZE-OPTION= is to convert a string option provided on the command-line, which represents a database connection string into an actual session object for the database.
The default implementation of this generic function simply returns the already set value, e.g. calls =#'IDENTITY= on the last derived value.
** CLINGON:DERIVE-OPTION-VALUE
The =CLINGON:DERIVE-OPTION-VALUE= is called whenever an option is provided on the command-line.
If that option accepts an argument, it will be passed the respective value from the command-line, otherwise it will be called with a =NIL= argument.
Responsibility of the option is to derive a value from the given input and return it to the caller. The returned value will be set by the parser and later on it will be used to produce a final value, by calling the =CLINGON:FINALIZE-OPTION= generic function.
Different kinds of options implement this one different -- for example the =:LIST= option kind accumulates each given argument, while others ignore any previously derived values and return the last provided argument.
The =:ENUM= option kind for example will derive a value from a pre-defined list of allowed values.
If an option fails to derive a value (e.g. invalid value has been provided) the implementation of this generic function should signal a =CLINGON:OPTION-DERIVE-ERROR= condition, so that =clingon= can provide appropriate restarts.
** CLINGON:OPTION-USAGE-DETAILS
This generic function is used to provide a pretty-printed usage format for the given option. It will be used when printing usage information on the command-line for the respective commands.
** CLINGON:OPTION-DESCRIPTION-DETAILS
This generic function is meant to enrich the description of the option by providing as much details as possible for the given option, e.g. listing the available values that an option can accept.
** CLINGON:MAKE-OPTION
The =CLINGON:MAKE-OPTION= generic function is the primary way for creating new options. Implementors of new option kinds should simply provide an implementation of this generic function, along with the respective option kind.
Additional option kinds may be implemented as separate sub-systems, but still follow the same principle by providing a single and consistent interface for option creation.
This section contains short guides explaining how to develop new options for =clingon=.
** Developing an Email Option
The option which we'll develop in this section will be used for specifying email addresses.
Start up your Lisp REPL session and do let's some work. Load the =:clingon= and =:cl-ppcre= systems, since we will need them.
CL-USER> (ql:quickload :clingon) CL-USER> (ql:quickload :cl-ppcre)
We will first create a new package for our extension and import the symbols we will need from the =:clingon= and =:cl-ppcre= systems.
(defpackage :clingon.extensions/option-email (:use :cl) (:import-from :cl-ppcre :scan) (:import-from :clingon :option :initialize-option :derive-option-value :make-option :option-value :option-derive-error) (:export :option-email)) (in-package :clingon.extensions/option-email)
Then lets define the class, which will represent an email address option.
(defclass option-email (option) ((pattern :initarg :pattern :initform "^[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+.[a-zA-Z0-9-.]+$" :reader option-email-pattern :documentation "Pattern used to match for valid email addresses")) (:default-initargs :parameter "EMAIL") (:documentation "An option used to represent an email address"))
Now we will implement =CLINGON:INITIALIZE-OPTION= for our new option. We will keep the default initialization logic as-is, but also add an additional step to validate the email address, if we have any initial value at all.
(defmethod initialize-option ((option option-email) &key) "Initializes our new email address option" ;; Make sure to invoke our parent initialization method first, so ;; various things like setting up initial value from environment ;; variables can still be applied. (call-next-method)
;; If we don't have any value set, there's nothing else to ;; initialize further here. (unless (option-value option) (return-from initialize-option))
;; If we get to this point, that means we've got some initial value, ;; which is either set as a default, or via environment ;; variables. Next thing we need to do is make sure we've got a good ;; initial value, so let's derive a value from it. (let ((current (option-value option))) (setf (option-value option) (derive-option-value option current))))
Next we will implement =CLINGON:DERIVE-OPTION-VALUE= for our new option kind.
(defmethod derive-option-value ((option option-email) arg &key) "Derives a new value based on the given argument. If the given ARG represents a valid email address according to the pattern we know of we consider this as a valid email address." (unless (scan (option-email-pattern option) arg) (error 'option-derive-error :reason (format nil "~A is not a valid email address" arg))) arg)
Finally, lets register our new option as a valid kind by implemeting the =CLINGON:MAKE-OPTION= generic function.
(defmethod make-option ((kind (eql :email)) &rest rest) (apply #'make-instance 'option-email rest))
We can test things out now. Go back to your REPL and try these expressions out. First we make a new instance of our new option.
(defparameter opt (make-option :email :short-name #\e :description "email opt" :key :email))
And now, lets validate a couple of good email addresses.
EXTENSIONS/OPTION-EMAIL> (derive-option-value opt "test@example.com") "test@example.com" EXTENSIONS/OPTION-EMAIL> (derive-option-value opt "foo@bar.com") "foo@bar.com"
If we try deriving a value from a bad email address we will have a condition of type =CLINGON:OPTION-DERIVE-ERROR= signalled.
EXTENSIONS/OPTION-EMAIL> (derive-option-value opt "bad-email-address-here") ; Debugger entered on #<OPTION-DERIVE-ERROR {1002946463}> ... bad-email-address-here is not a valid email address [Condition of type OPTION-DERIVE-ERROR]
Good, we can catch invalid email addresses as well. Whenever an option fails to derive a new value from a given argument, and we signal =CLINGON:OPTION-DERIVE-ERROR= condition we can recover by providing new values or discarding them completely, thanks to the Common Lisp Condition System.
Last thing to do is actually package this up as an extension system and register it in Quicklisp. That way everyone else can benefit from the newly developed option.
=clingon= provides support for Bash and Zsh shell completions.
** Bash Completions
In order to enable the Bash completions for your =clingon= app, follow these instructions.
Depending on your OS you may need to install the =bash-completion= package. For example on Arch Linux you would install it like this.
sudo pacman -S bash-completion
Then source the completions script.
APP=app-name source extras/completions.bash
Make sure to set =APP= to your correct application name.
The [[https://github.com/dnaeon/clingon/blob/master/extras/completions.bash][completions.bash]] script will dynamically provide completions by invoking the =clingon= app with the =--bash-completions= flag. This builtin flag when provided on the command-line will return completions for the sub-commands and the available flags.
** Zsh Completions
When developing your CLI app with =clingon= you can provide an additional command, which will take care of generating the Zsh completion script for your users.
The following code can be used in your app and added as a sub-command to your top-level command.
(defun zsh-completion/command () "Returns a command for generating the Zsh completion script" (clingon:make-command :name "zsh-completion" :description "generate the Zsh completion script" :usage "" :handler (lambda (cmd) ;; Use the parent command when generating the completions, ;; so that we can traverse all sub-commands in the tree. (let ((parent (clingon:command-parent cmd))) (clingon:print-documentation :zsh-completions parent t)))))
You can also check out the =clingon-demo= app for a fully working CLI app with Zsh completions support.
[[./images/clingon-zsh-completions.gif]]
The demo =clingon= apps from this repo are usually built using [[https://asdf.common-lisp.dev/][ASDF]] with =:build-operation= set to =program-op= and the respective =:entry-point= and =:build-pathname= specified in the system definition. See the included =clingon.demo.asd= and =clingon.intro.asd= systems for examples.
You can also use [[https://www.xach.com/lisp/buildapp/][buildapp]] for building the =clingon= apps.
This command will build the =clingon-demo= CLI app using =buildapp=.
$ buildapp \ --output clingon-demo \ --asdf-tree ~/quicklisp/dists/quicklisp/software/ \ --load-system clingon.demo \ --entry main \ --eval '(defun main (argv) (let ((app (clingon.demo::top-level/command))) (clingon:run app (rest argv))))'
Another approach to building apps using =buildapp= is to create a =main= entrypoint in your application, similarly to the way you create one for use with ASDF and =:entry-point=. This function can be used as an entrypoint for [[https://www.xach.com/lisp/buildapp/][buildapp]] apps.
(defun main (argv) "The main entrypoint for buildapp apps" (let ((app (top-level/command))) (clingon:run app (rest argv))))
Then build your app with this command.
$ buildapp \ --output my-app-name \ --asdf-tree ~/quicklisp/dists/quicklisp/software/ \ --load-system my-system-name \ --entry my-system-name:main
** Additional Documentation Generators
As of now =clingon= supports generating documentation only in /Markdown/ format.
Would be nice to have additional documentation generators, e.g. /man pages/, /HTML/, etc.
** Performance Notes
=clingon= has been developed and tested on a GNU/Linux system using SBCL.
Performance of the resulting binaries with SBCL seem to be good, although I have noticed better performance when the binaries have been produced with Clozure CL. And by better I mean better in terms of binary size and speed (startup + run time).
Although you can enable compression on the image when using SBCL you have to pay the extra price for the startup time.
Here are some additional details. Build the =clingon-demo= app with SBCL.
$ LISP=sbcl make demo sbcl --eval '(ql:quickload :clingon.demo)' \ --eval '(asdf:make :clingon.demo)' \ --eval '(quit)' This is SBCL 2.1.7, an implementation of ANSI Common Lisp. More information about SBCL is available at http://www.sbcl.org/.
SBCL is free software, provided as is, with absolutely no warranty. It is mostly in the public domain; some portions are provided under BSD-style licenses. See the CREDITS and COPYING files in the distribution for more information. To load "clingon.demo": Load 1 ASDF system: clingon.demo ; Loading "clingon.demo" [package clingon.utils]........................... [package clingon.conditions]...................... [package clingon.options]......................... [package clingon.command]......................... [package clingon]................................. [package clingon.demo] [undoing binding stack and other enclosing state... done] [performing final GC... done] [defragmenting immobile space... (fin,inst,fdefn,code,sym)=1118+969+19070+19610+26536... done] [saving current Lisp image into /home/dnaeon/Projects/lisp/clingon/clingon-demo: writing 0 bytes from the read-only space at 0x50000000 writing 736 bytes from the static space at 0x50100000 writing 31391744 bytes from the dynamic space at 0x1000000000 writing 2072576 bytes from the immobile space at 0x50200000 writing 12341248 bytes from the immobile space at 0x52a00000 done]
Now, build it using Clozure CL.
$ LISP=ccl make demo ccl --eval '(ql:quickload :clingon.demo)' \ --eval '(asdf:make :clingon.demo)' \ --eval '(quit)' To load "clingon.demo": Load 1 ASDF system: clingon.demo ; Loading "clingon.demo" [package clingon.utils]........................... [package clingon.conditions]...................... [package clingon.options]......................... [package clingon.command]......................... [package clingon]................................. [package clingon.demo].
In terms of file size the binaries produced by Clozure CL are smaller.
$ ls -lh clingon-demo* -rwxr-xr-x 1 dnaeon dnaeon 33M Aug 20 12:56 clingon-demo.ccl -rwxr-xr-x 1 dnaeon dnaeon 45M Aug 20 12:55 clingon-demo.sbcl
Generating the Markdown documentation for the demo app when using the SBCL executable looks like this.
$ time ./clingon-demo.sbcl print-doc > /dev/null
real 0m0.098s user 0m0.071s sys 0m0.027s
And when doing the same thing with the executable produced by Clozure CL we see these results.
$ time ./clingon-demo.ccl print-doc > /dev/null
real 0m0.017s user 0m0.010s sys 0m0.007s
The =clingon= tests are provided as part of the =:clingon.test= system.
In order to run the tests you can evaluate the following expressions.
CL-USER> (ql:quickload :clingon.test) CL-USER> (asdf:test-system :clingon.test)
Or you can run the tests using the =run-tests.sh= script instead, e.g.
LISP=sbcl ./run-tests.sh
Here's how to run the tests against SBCL, CCL and ECL for example.
for lisp in sbcl ccl ecl; do echo "Running tests using ${lisp} ..." LISP=${lisp} make test > ${lisp}-tests.out done
A few Docker images are available.
Build and run the tests in a container.
docker build -t clingon.test:latest -f Dockerfile.sbcl . docker run --rm clingon.test:latest
Build and run the =clingon-intro= application.
docker build -t clingon.intro:latest -f Dockerfile.intro . docker run --rm clingon.intro:latest
Build and run the =clingon.demo= application.
docker build -t clingon.demo:latest -f Dockerfile.demo . docker run --rm clingon.demo:latest
=clingon= is hosted on [[https://github.com/dnaeon/clingon][Github]]. Please contribute by reporting issues, suggesting features or by sending patches using pull requests.
This project is Open Source and licensed under the [[http://opensource.org/licenses/BSD-2-Clause][BSD License]].
Authors
Marin Atanasov Nikolov dnaeon@gmail.com