Added support for flag arguments with script defn

[frenchy64]

Here is an attempt to tie in command line arguments to the 'defn script form.

The parameter list must contain 0-n symbols followed by 0-n vectors.

This uses an external library shFlags, it must be source'd before use. It is probably best to leave the user to do this.

(script 
  ;;; foo 1 2 --example something => something
  (defn foo [x y 
             [:string "example" "e" "doc" "default"]] 
    (echo @FLAGS_example)))

  ;;; bar --example something => something
  (defn bar [[:string "example" "e" "doc" "default"]] 
    (echo @FLAGS_example))

  ;;; bar --example something -s yes => yes
  (defn foo [[:integer "something" "s" "doc" 122]
             [:string "example" "e" "doc" "default"]] 
    (bar @FLAGS_something))

Ideas for expansion:

• Use --long and -s in the arguments like

[:integer --long -l "Doc" "default"]

[hugoduncan]

There is a precedent for an external lib, as stevedore uses hashlib which is included in the project resources. I'm not sure that this is the best way of doing things though, since it freezes the lib version.

Isn't the main use for shFlags for top level script arguments? how would top level arguments be supported by this?

[frenchy64]

Yes, shFlags is most useful with top level arguments. This just solves a small niche I currently desire.

For top level arguments, I would wrap the arguments in a function.

(script
  (declare-arguments
    [x y w
     [:string "example" "e" "doc" "default"]
     [:integer "another" "a" "this" "here"]])

  (echo x)
  (echo FLAGS_example))

It presents a good opportunity to abstract out the argument parsing from defn.

Another possibility, slightly more drastic, would be to change script.

(script 
  [x y w
   [:string "example" "e" "doc" "default"]
   [:integer "another" "a" "this" "here"]]

  (echo x)
  (echo FLAGS_example))

I have some reservations about this though. Clearly issues of composability crop up with things like do-script.

Maybe a higher level with-arguments is appropriate.

(with-arguments 
  [x y w
   [:string "example" "e" "doc" "default"]
   [:integer "another" "a" "this" "here"]])
  (do-script
    (script 
      (echo FLAGS_example))
    (script 
      (echo x))

Interested in hearing your thoughts on these.

As for the issue of external libraries, I'm not sure what the best way to approach it would be.

Should we make it easy to use a custom version, say with a *shflags-path* value? Would this even be useful?

[hugoduncan]

I like the declare-arguments idea, and think that should be included as a baseline. If I understand correctly, this can be implemented as a script function rather that requiring modification to stevedore. The proposed defn change would be more attractive if it can follow clojure syntax somehow (not sure that is possible).

Modifying stevedore/script is not really on the cards, as it has a clear purpose at the moment, which is completely orthogonal to argument parsing.

with-arguments also seems problematic - how would you now whether to apply it to the script forms in your example?

I think that bundling a know working version is a good idea. It is still up to the user to include it in the generated script, so they remain free to do so using other means.

[frenchy64]

I like the declare-arguments idea, and think that should be included as a baseline.

I agree, declare-arguments has much more utility. I think you've outlined the reasons why the others would cause trouble.

If I understand correctly, this can be implemented as a script function rather that requiring modification to stevedore.

Yes, that is correct. Hence I will preface declare-arguments with ~.

The proposed defn change would be more attractive if it can follow clojure syntax somehow (not sure that is possible).

Me neither, but lets have a shot! Here is a gradual evolution of ideas.

We could take some inspiration from with-command-line and add a docstring argument. shFlags has some support for this.

(script
  (~declare-arguments
    ~(str (apply interpose "\n"
                 "This is an example program"
                 "If you need some additional help, try the man page"))
    [x y w
     [:string "example" "e" "doc" "default"]
     [:integer "another" "a" "this" "here"]])

We can steal the syntax for flag declarations from with-command-line.

(script
  (~declare-arguments
    ~(str (apply interpose "\n"
                 "This is an example program"
                 "If you need some additional help, try the man page"))
    [x y w
     [example e "This is the documentation string" "Something"] ;; changed declarations
     [another a "Here is another doc string" "Defaultvalue"]]))

The docstring could be applied to defn. Would have to be between the function name and args. Not sure about anonymous functions yet. This example also uses flags like with-command-line:

(script
  (defn example
    "This is an example program" ;; docstring
    [x y w
     [example e "This is the documentation string" "Something"]
     [another a "Here is another doc string" "Defaultvalue"]]

    (echo example)
    (echo x))

  (example --example "blah" 1 2 3))

The type "declarations" in my initial proposal are just shFlags making sure things "look" like a number or string. Could default to String but give an optional Clojuresue type hints like:

(script
  (defn example
    "This is an example program"
    [x y w
     [test t "This is the documentation string" "Something"]
     [^String we w "This is the documentation string" "Something"]
     [^Boolean example e "This is the documentation string" false]
     [swith s "This is the documentation string"] ;;; omitting default value implies boolean flag switch
     [^Integer another a "Here is another doc string" 12]]

    (echo example)
    (echo x))

  (example --example --we "blah" 1 2 3))

For now I'm assuming an implementation is possible, I haven't actually attempted with the type hints and docstrings.

Thoughts?

I think that bundling a know working version is a good idea. It is still up to the user to include it in the generated script, so they remain free to do so using other means.

So provide a script like (~include-shflags)? We would manage the path to our bundled shFlags version.

[hugoduncan]

The proposed defn change would be more attractive if it can follow
clojure syntax somehow (not sure that is possible).

Me neither, but lets have a shot! Here is a gradual evolution of ideas.

We could take some inspiration from [with-command-line] [1] and add a
docstring argument. shFlags has [some support] [2] for this.

Adding a doc string seems to be a good idea, independent of direct support
in shFlags. For me this begs the question of how (human) readable the
generated code should be.

We can steal the syntax for flag declarations from with-command-line.

Given the similarity anyway, this sounds like a good idea.

The type "declarations" in my initial proposal are just shFlags making
sure things "look" like a number or string. Could default to String but
give an optional Clojuresue type hints like:

Using type hints looks good. The alternative could be to use keywords, but
make them an optional last element of the vector.

Another possibility would be to use map destructuring for the arguments:

(script
   (defn example
     "This is an example program"
     [x y w & {:keys [test we example switch another]
               :or {test "Something"
                    we "Something"
                    example false
                    another 12}}
     (echo example)
     (echo x))

   (example --example --we "blah" 1 2 3))

This looses information, spreads related information, and increases
duplication, but looks more familiar. I wish clojure had a standard way
of documenting keyword arguments… A clojure defn form that generated a
suffix to the docstring based on something like the declare-arguments
argument vectors might be useful (i.e. similar to your proposed script
defn).

I think that bundling a know working version is a good idea. It is
still up to the user to include it in the generated script, so they
remain free to do so using other means.

So provide a script like (~include-shflags)? We would manage the path
to our bundled shFlags version.

I see a possible advantage to having this as a script function, to allow
possibly different implementation on windows. For hashlib I just defined
a var stevedore/hashlib.

Hugo Duncan

[frenchy64]

For now the keyword "types" are good enough. True Clojure type declarations will be more involved to extract, as you need to get to them before the evaluator, in a macro.

My recent commits have settled on this:

(spit "test.sh"
  (with-script-context [:default]
    (stevedore/script
      ". \"/home/ambrose/.cljr/bin/shflags\""

      (~declare-arguments
         "Test"
         [movie
          [:integer days d "Number of days in movie" 1]
          [:string subtitle s "Subtitle of the movie" "Dooomm"]])

      (echo @movie @days @subtitle)
      (defn foo
        [a]
        (echo @a)))))

=>

. "/home/ambrose/.cljr/bin/shflags"

FLAGS_HELP="Test"
DEFINE_integer "days" "1" "Number of days in movie" "d"
DEFINE_string "subtitle" "Dooomm" "Subtitle of the movie" "s"
FLAGS "$@" || exit 1
eval set -- "${FLAGS_ARGV}"

movie=$1
echo ${movie} ${days} ${subtitle}

foo() {
   a=$1
   echo ${a}
}

...with some manual indenting/newlines.

I also settled on the C-like function declarations, apparently more portable.