dnaeon / clingon

Command-line options parser system for Common Lisp
Other
122 stars 7 forks source link

How can I add a new option type like Python click's nargs? #2

Closed Liutos closed 2 years ago

Liutos commented 2 years ago

In Python there's a library named click, it can handles options required more than one values, like followed

@click.command()
@click.option('--pos', nargs=2, type=float)
def findme(pos):
    a, b = pos
    click.echo(f"{a} / {b}")

and the usage

$ findme --pos 2.0 3.0
2.0 / 3.0

How can I control the number of arguments consume by an option in clingon? Seems need to dig into implementation of generic function parse-option.

dnaeon commented 2 years ago

Hey @Liutos ,

Currently clingon does not have support for nargs-like options as you can find in click.

Also, I'm not really sure whether nargs is a good design decision to have be honest. Take for example the following command line.

$ ./foo.py --n 1 --pos --n 42

Just by looking at the command-line it's impossible to tell what is the type of --pos and what is the final value of --n. It looks like --pos is a flag, but it is not. Also, the final value of --n is not 42, but 1.

Here's the code with click version 8.1.3.

#!/usr/bin/env python

import click

@click.command()
@click.option('--n', default=1)
@click.option('--pos', nargs=2, type=str)
def findme(n, pos):
    a, b = pos
    click.echo(f"a: {a}, b: {b}, n: {n}")

if __name__ == '__main__':
    findme()

In my opinion using nargs might lead to ambiguity, and for that reason I try to avoid using it. That's just my personal opinion, though :)

The same behaviour can definitely be implemented in clingon as well, but would require changes to the parse-option methods, which need to consume the next argument (up to nargs), although I'm not really convinced that's a good idea, because of the ambiguity reasons, I've mentioned above.

Below, I'm describing a few approaches you might want to take in order to implement a similar behaviour.

Approach A

Here's an alternative approach you can take when using clingon -- you can create a new custom option, which handles that one for you.

The documentation also provides an example of developing an email-based option: https://github.com/dnaeon/clingon#developing-new-options

Here's an example of how to develop an option which may represent the size of an item. Different dimensions are also supported by specifying the dimensions initarg. Values may be integer or a float.

Start up your Lisp REPL and load the parse-float and str systems.

CL-USER> (ql:quickload :parse-float)
To load "parse-float":
  Load 1 ASDF system:
    parse-float
; Loading "parse-float"

(:PARSE-FLOAT)
CL-USER> (ql:quickload :str)
To load "str":
  Load 1 ASDF system:
    str
; Loading "str"
...
(:STR)

Create a new package for our custom option, import the required symbols from clingon, and then switch to the new package.

(defpackage :clingon.extensions/option-size
  (:use :cl)
  (:import-from :str)
  (:import-from :parse-float)
  (:import-from
   :clingon
   :option
   :initialize-option
   :derive-option-value
   :make-option
   :option-value
   :option-derive-error)
  (:export
   :option-position))
(in-package :clingon.extensions/option-size)

We'll add a couple of utility functions for converting string values to integer and float. Note, that the parse-integer-or-lose function is also part of the clingon.options package, but is not exported at the time of writing this.

(defun parse-integer-or-lose (value &key (radix 10))
  (when (integerp value)
    (return-from parse-integer-or-lose value))
  (let ((int (parse-integer value :radix radix :junk-allowed t)))
    (unless int
      (error 'option-derive-error :reason (format nil "Cannot parse ~A as integer" value)))
    int))

(defun parse-float-or-lose (value &key (radix 10))
  (when (floatp value)
    (return-from parse-float-or-lose value))
  (let ((f (parse-float:parse-float value :radix radix :junk-allowed t)))
    (unless f
      (error 'option-derive-error :reason (format nil "Cannot parse ~A as float" value)))
    f))

This is what our custom option looks like.

(defclass option-size (option)
  ((dimensions
    :initarg :dimensions
    :initform (error "Must specify size dimensions")
    :reader option-size-dimensions
    :documentation "The size dimensions")
   (separator
    :initarg :separator
    :initform #\x
    :reader option-size-separator
    :documentation "Separator to use")
   (type
    :initarg :type
    :initform :int
    :reader option-size-type
    :documentation "Type of the values to parse, e.g. :int or :float")
   (supported-types
    :initform '(:int :float)
    :reader option-size-supported-types
    :documentation "Supported types for the parsed values")
   (radix
    :initarg :radix
    :initform 10
    :reader option-size-radix))
  (:default-initargs
   :parameter "SIZE")
  (:documentation "An option used to represent a size, e.g. 10x20x30"))

We need to define how we initialize the option as well.

(defmethod initialize-option ((option option-size) &key)
  "Initializes our new SIZE address option"
  ;; Make sure to invoke our parent initialization method first, so
  ;; various things like setting up initial value from environment
  ;; variables can still be applied.
  (call-next-method)

  ;; If we don't have any value set, there's nothing else to
  ;; initialize further here.
  (unless (option-value option)
    (return-from initialize-option))

  ;; If we get to this point, that means we've got some initial value,
  ;; which is either set as a default, or via environment
  ;; variables. Next thing we need to do is make sure we've got a good
  ;; initial value, so let's derive a value from it.
  (let ((current (option-value option)))
    (setf (option-value option)
          (derive-option-value option current))))

Then the method which derives values is implemented as such.

(defmethod derive-option-value ((option option-size) arg &key)
  "Derives a new value based on the given argument."
  (let* ((separator (option-size-separator option))
         (dimensions (option-size-dimensions option))
         (opt-type (option-size-type option))
         (supported-types (option-size-supported-types option))
         (radix (option-size-radix option))
         (items (str:split separator arg)))
    (unless (member opt-type supported-types)
      (error 'option-derive-error :reason (format nil "Unsupported type ~A" opt-type)))
    (unless (= dimensions (length items))
      (error 'option-derive-error :reason (format nil "Bad number of values for size dimension(s) of ~A" dimensions)))
    (ecase opt-type
      (:int (mapcar
             (lambda (value)
               (parse-integer-or-lose value :radix radix))
             items))
      (:float (mapcar
               (lambda (value)
                 (parse-float-or-lose value :radix radix))
               items)))))

Finally, we register our new option.

(defmethod make-option ((kind (eql :size)) &rest rest)
  (apply #'make-instance 'option-size rest))

Now we can test out our new option. Switch to the newly created package.

CL-USER> (in-package :clingon.extensions/option-size)
#<PACKAGE "CLINGON.EXTENSIONS/OPTION-SIZE">

... and create a new option of size kind.

EXTENSIONS/OPTION-SIZE> (defparameter *opt*
              (make-option :size
                       :short-name #\s
                       :long-name "size"
                       :description "size of an item"
                       :dimensions 2
                       :key :size))
*OPT*

The option we've created above represents the size of an item with 2 dimensions. Now we can derive values from it, e.g.

EXTENSIONS/OPTION-SIZE> (derive-option-value *opt* "10x20")
(10 20)

EXTENSIONS/OPTION-SIZE> (derive-option-value *opt* "42x42")
(42 42)

You could probably extend the OPTION-SIZE class further and add slots to represent the unit, if needed. This can become really useful when you generate the autocompletions or documentation for the respective option.

Approach B

The nargs behaviour in click seem to be similar to the :list option kind in clingon. The only noticeable difference is that in click you have to provide all arguments immediately after the option, e.g. --foo one two, while in clingon this would usually be written on the command-line as --foo one --foo bar.

For example.

(defun top-level/handler (cmd)
  (let ((items (clingon:getopt cmd :items)))
    (format t "Number of items: ~A" (length items))
    (format t "The items are: ~A" items)))

(defun top-level/options ()
  "Returns the top-level command options"
  (list
   (clingon:make-option
    :list
    :short-name #\i
    :long-name "item"
    :description "item to add"
    :key :items)))

(defun top-level/command ()
  (clingon:make-command
   :name "foo"
   :description "some foo command"
   :options (top-level/options)))

We can test it out.

CL-USER> (defparameter *cmd* (top-level/command))
*CMD*
CL-USER> (defparameter *command-line-args* (list "--item" "one" "--item" "two"))
*COMMAND-LINE-ARGS*
CL-USER> (clingon:parse-command-line *cmd* *command-line-args*)
#<CLINGON.COMMAND:COMMAND name=foo options=4 sub-commands=0>
CL-USER> (clingon:getopt *cmd* :items)
("one" "two")
T
CL-USER> (top-level/handler *cmd*)
Number of items: 2
The items are: (one two)
NIL

An improvement to the code above would be a simple customized version :list option kind, which enforces the number of arguments, which the option must have during finalization, which is done using the FINALIZE-OPTION method.

Approach C

The last approach that comes to my mind is to add nargs-like support in the parser. That could be implemented directly in clingon, or could be an external system, which customizes the parser behaviour as we have it right now in clingon.

Example code would look like this (note, I haven't tested this one, and is incomplete, but it should show you the idea).


(defclass custom-command (clingon:command)
  ()
  (:documentation "A customized clingon command"))

(defclass custom-option (clingon:option)
  ((nargs
    :initarg :nargs
    :initform 1
    :reader option-nargs
    :documentation "Number of arguments an option consumes")))

(defmethod parse-option ((kind (eql :long)) (command custom-command) &key)
  "Parses a long option"
  (let* ((arg (pop (command-args-to-parse command)))
         (equals-position (position #\= arg))
         (long-name (subseq arg 2 equals-position))
         (long-name-full (format nil "--~A" long-name))
         (option (find-option :long command long-name))
         (optargs nil))
    ;; Unknown option
    (unless option
      (handle-unknown-option-with-restarts command kind long-name-full)
      ;; We are done here, let the parser handle any new input on the
      ;; next iteration.
      (return-from parse-option))

    ;; Valid option
    (setf (option-is-set-p option) t)
    ;; Option takes a parameter, make sure we've got an argument from
    ;; which to derive a value and set `optargs' accordingly
    (when (option-parameter option)
      ;; Option takes a parameter
      (if equals-position
          (setf optargs (subseq arg (1+ equals-position))) ;; --arg=foo
          ;;
          ;; !! New code goes here, which collects NARGS arguments !!
          ;;
          (setf optargs
                (loop :repeat (option-nargs option)
                      :collect (pop (command-line-args-to-parse command))))) ;; --arg foo
      ;; Handle missing argument for the option
      (when (or (null optargs) (string= optarg ""))
        (setf optargs (handle-missing-argument-with-restarts command option))
        (unless optargs
          (return-from parse-option))))

    (derive-option-with-restarts command option optarg)))

With this approach derive-option-with-restarts method should be updated as well, or it could be done in the method above to take into account the number of arguments that need to be consumed.

Conclusion

There are number of ways to implement a similar behaviour, so it's up to you which one works best.

Personally, I would go with a custom option modeled around the data that the option will work with, as that would allow me to express the semantics in a more concise way.

Hopefully I managed to give you some ideas how to approach this one. Thanks!

Liutos commented 2 years ago

That's awesome, thanks for your 2 excellent approaches.

Yes, as you said, the nargs in click seem ambiguous, and if needed, I prefer to use your approach A. At the beginning, I try to avoid introducing some ad-hoc syntax, like the times sign x in the example above, then I try to simulate click's nargs. But right now, may be an explicit syntax is a better one.

Actually, the second approach is a more direct solution, I'm just curious whether I can customize a subclass of option for simulating nargs.

Anyway, thank you very much ;)