Easy way to define wrappers

[Metaxal]

One often needs to define wrappers around existing functions, passing some of the same default arguments, either positional or keyword based. For example, if one wants to define a curried version of dict-ref, one has to redefine the default argument for failure:

(define ((curried-dict-ref key [default <failure-result-tbd>]) a-dict)
  (dict-ref a-dict key default))

Not only is this cumbersome, error-prone and against DRY, but the docs are actually not helpful to define the correct default value.

Hence I would very much like in Rhombus a global mechanism for easily defining wrappers around existing functions with default arguments.

[sorawee]

Personally, I don't like positional optional arguments, especially when there are more than one of them. One problem is exactly what you mentioned above. Another problem is that if users want to provide a value at the nth argument, they are forced to provide all arguments before the nth argument too.

So at least for me, I won't consider the above case.

However, the problem does exist for keyword arguments too. But for that, see https://github.com/racket/rhombus-brainstorming/issues/66#issuecomment-513924762.

[Metaxal]

99.99 agree with you and 100% with https://github.com/racket/rhombus-brainstorming/issues/66#issuecomment-513924762 !

The remaining 0.01 is because I'm okay with single optional positional arguments, so I'd still like something for the case above. However if we decide that all optional arguments are keyword-based (which may make things simpler), I'm okay with this too.

(I'm also not okay with more than 4 positional (mandatory) arguments.)

[jackfirth]

Is there a way to solve the problem of wrapping functions with default arguments without introducing runtime overhead?

Personally I think this is more of an API design problem than a language feature problem. The example you gave wouldn't be an issue if dict-ref returned an option value instead of accepting a failure thunk. If a default argument is complex enough to make wrapping the function difficult, I think using a simpler default would be best.

[Metaxal]

[This is a little beside the point but: With option values you're changing the meaning of dict-ref. Raising an exception on absent key is a perfectly valid design, which I believe is more useful than forcing the user to check for the value of option key and throwing an exception manually, in particular because the design allows you to replace the exception raising with a default value, thus allowing you to not having to handle exceptions.]

I believe it is feasible to define a large class of wrappers without introducing unnecessary overhead, since for most functions defined with (define (foo args ...) body ...) it is possible to know their keyword arguments at compile/read time.

[jackfirth]

I think (present-value (dict-ref-option dict key)) and (option-get (dict-ref-option dict key) default) aren't that much different from (dict-ref dict key) and (dict-ref dict key default), and they get around the complexity of wrapping dict-ref with something that propagates a default failure handler correctly.

[Metaxal]

(First I feel pretty differently about "aren't that much different": it doubles the code. I don't want to have to write 2 pages of code when I can write 1.)

More importantly, I picked a very simple example for the sake of clarity. But an example that bothers me much more is wrapping around the plot function for example, which has many optional keyword arguments.

[rocketnia]

I sympathize with this issue. I think it could be approached like this:

• In an argument list, allow some kind of syntax that binds an optional argument to an option value, rather than requiring a default value to be specified.
• In a function call, allow some kind of syntax to pass along an option value where an optional argument is expected.

For instance:

(define ((curried-dict-ref key [#? default]) a-dict)
  (dict-ref a-dict key [#? default]))

[Metaxal]

Here's a proof-of-concept macro that implements a slight variation of this idea, except that:

• function calls are just standard racket calls (no #? in calls)
• optional values are not wrapped in an option struct, so this should cooperate more nicely with the compiler
• instead a no-value value is used as a placeholder
• an argument #:? [var val] defines a standard keyword argument #:var [var val].
• at a call site, if the keyword is not given or if the value no-value is passed, var is bound to val
• #:? var is the same as #:? [var no-value]

Any major defect to this approach? (of course this should be generalized to lambda, non-optional keywords, possibly positional default arguments—or maybe not)

[Metaxal]

And, if plot were implemented with this define*, an example wrapper my-plot that just redefines the default values for x-min and y-min---while all other default values are retained---would look like this:

(define* (my-plot renderer-tree
                  #:? [x-min 0]
                  #:? x-max
                  #:? [y-min 0]
                  #:? y-max
                  #:? width
                  #:? height
                  #:? title
                  #:? x-label
                  #:? y-label
                  #:? legend-anchor
                  #:? out-file
                  #:? out-kind)
  (plot renderer-tree
        #:x-min x-min
        #:x-max x-max
        #:y-min y-min
        #:y-max y-max
        #:width width
        #:height height
        #:title title
        #:x-label x-label
        #:y-label y-label
        #:legend-anchor legend-anchor
        #:out-file out-file
        #:out-kind out-kind))

which is pretty much what I'm looking for.

[Metaxal]

I've made a package that can be straightforwardly used with Racket, as a drop-in replacement for define and lambda. All the bells and whistles (curried define, keywords, good error messages which are mostly consistent with the current define and lambda) of both forms should work as expected since I adapted from syntax/parse/lib/function-header (although this lib has a bug regarding arg order in keyword args).

Feedback welcome.

[rocketnia]

@Metaxal I'm really glad you arrived somewhere you like. Incidentally, I don't think it's just a slight variation on what I proposed; I was basically saying "here's how we can get optional arguments to dovetail into option values" and you dropped the option values. :-p More to the point though, I think there might be a "major defect" I see in your approach, depending on what scenarios you consider important.

You have a no-value value that has special behavior as a function argument, and I'm concerned this would be error-prone. This value is designed to let arguments be passed along, so I expect someday I'll want to store it somewhere for the time being and then pass it along later, or transform it first and then pass it along. Unfortunately, if I intend to store it by passing it to some data constructor or some database method, or if I intend to transform it first by passing it to some function, then technically I'm passing it as an argument to something, so I might set off its special behavior immediately. Ideally, I wouldn't want its special behavior to activate until I was actually passing it to its intended destination.

From the perspective of someone trying to supply a well-behaved data constructor, database method, or argument transformer, there's a distinction to be made between "a no-value passed to me to say my argument should have its default value" and "a no-value passed to me for me to pass through to someone else." If these are both represented by no-value, it's difficult to distinguish.

There may be workarounds. For instance, the author of that data constructor, database method, or argument transformer might opt for a design that doesn't use optional arguments itself, so that it doesn't have to represent the first case and can focus on representing the second case.

A lot of languages rely on workarounds like those, and people still use them, so it's hard to call it a "major defect." Personally, each time I have to use a quirky workaround for this, especially when that workaround is to stop using optional arguments in some part of my code, I feel like I'm in a language that doesn't fully support optional arguments.

That's the same feeling I can sometimes get in Racket when it's hard to write a wrapper or a contract for a function that has optional arguments. (And you're concerned about wrappers, so maybe my concern about this is relevant to you.) When optional arguments cause headaches in any of these ways, it's not the end of the world, because a language that doesn't support optional arguments is still a usable enough language. It just means that I tend to use the language in ways that avoid using optional arguments, or in ways that represent them longhand using explicit option values.

With option values, it's straightforward to distinguish between "an argument not passed to me" and "an argument I'm being told not to pass to someone else"; their representations are something like (nothing) and (just (nothing)). So I suppose my proposal for you now would be the same as before, in the sense that I'd bring back the option values and make them work in the same way (#?).

As far as other feedback goes, I notice you have #:? mean a certain thing in formal argument lists. Are you planning to customize #%app so that it can mean the same thing in a function call? That seems like it could shave off another 1/3 of the verbosity of your plot example.

[AlexKnauth]

Part of @rocketnia's proposal is a way to call functions using options to determine whether to use the default or not, but one question that arises is: what does it do if a later opitional-argument is supplied but an earlier one is not?

For example a function with 2 positional optional arguments

(define (f [a 1] [b 2]) (list a b))

In existing-Racket it's impossible to supply b without supplying a, but with rocketnia's proposal:

(f [#? nothing] [#? (just 5)])

If these were keyword arguments it would obviously evaluate to (list 1 5), but with positional arguments there's no existing way to do it... unless we change function values in a more fundamental way

[rocketnia]

That's right, and I don't want to give the wrong impression that it's a simple proposal.

I think it's similar in scope to keyword arguments. There would be a new apply like keyword-apply, a new procedure constructor like make-keyword-procedure, a new #%app, and a new lambda. I imagine there would be a new define that looked like it defined a function but actually defined a macro that could report arity errors at compile time and possibly expand to more efficient function calls when it was invoked in certain (sufficiently second-class) ways. If I understand correctly, these are all facets of the way keyword arguments currently work.

[Metaxal]

@rocketnia Thanks for the feedback, that's quite helpful.

Slight variation: That's right, it's not a slight variation, and it doesn't convey the meaning you intended—apologies for this approximation. What I meant to say is that it all clicked for me by reading your post, and I wanted to give you the credit for that. In particular, pass-through values can work only with keyword-based arguments (as mentioned by @AlexKnauth ), and using a special keyword such as #:? makes it easy to write the machinery. I'm comfortable with not using pass-through values with positional arguments. I'm even happy for Rhombus to not have optional positional arguments at all (@sorawee is already on board), in particular with a syntax as light as prefixing with #:?.

Storing no-value: I believe this one is a no-brainer: Just define a special (struct no-value-container (no-value)) (define no-value-contained (no-value-container no-value)), and pass this no-value-contained around for temporary storage. I fail to see how that's different from (nothing) and (just (nothing)) as you mention for option values. I could add this to the library to make it standard, but as of now I don't have/see a real use case, so I'll wait to see one of these to refine the idea—please do open an issue if you do. Furthermore, If you store a 'default-value' thing inside a database, you still have to invent a special token anyway that can be represented in the database format. Option values in Racket/Rhombus won't save you anything here either.

Major defect: I do see the the runtime overhead of option values as a major defect however, in particular because it would likely prevent the compiler from performing many optimizations. By contrast, my implementation really has next to zero runtime overhead, and should play well with the compiler.

Redefining #%app: I don't currently intend to do this because this requires creating keywords from already-bound identifiers so there's a tension between the callee's keywords and the user's ids, whereas in define the identifier is just being created so there's no tension. It's fine for wrappers, but not really for the more general case which could lead to bad styles such as (plot renderers #:? x-min #:x-max argh-I-wish-I-could-call-this-one-x-max ...). However, I'm currently working on a define-wrapper form that would shave off more than half of the verbosity:

(define-wrapper my-plot plot
  ; all these arguments are passed to plot
  (2d-renderers  ; positional argument
   #:? [x-min 0] ; redefine the default value
   #:? x-max     ; keep the original default value
   #:? [y-min 0]
   #:? y-max
   #:? width
   #:? height
   #:? title
   #:? x-label
   #:? y-label
   #:? legend-anchor
   #:? out-file
   #:? out-kind)
  ; these arguments are specific to my-plot
  (#:! count)
  ; optional body before calling plot
  (displayln (list x-min x-max))
  ; plot is being called around here
  #:with-result res
  ; optional body after calling plot
  (displayln (make-list count res)) ; whatever
  res)

(Comments welcome on this as I'm not really settled yet) If Rhombus has the good idea of storing functions' headers for more introspection, wrappers would even just need to specify the redefined keywords only.

[rocketnia]

Thanks for hearing my concerns. Those are thoughtful answers.

The workaround I would probably use is an immutable box. You don't find uses for an immutable box every day. 😛

That's a fascinating define-wrapper DSL. Maybe it could define some kind of local variable call-wrapped-function so that you can write (define res (call-wrapped-function)) where you currently have #:with-result res. I think that would make it a little easier to remember how to write the different parts of the wrapper body.

The argument lists might be clearer if they're shaped something like (define-wrapper (my-plot (plot 2d-renderers <...> #:? out-kind) #! count) <...>) so that they're associated with the names of the functions they'll be passed to.

Maybe the name of call-wrapped-function would be specified somewhere in those argument lists too. Depending on the syntax for that, it might lead down a path where the wrapper can wrap more than one function and merge their argument lists.

[Metaxal]

These are very useful comments, thanks!

Here's an example with the (implemented) refined design. This code:

(define (foo #:! a #:? [b 3] #:? [c 2])
  (values a b c))

(define-wrapper (bar [foo #:! a #:? b #:? [c 'cc]]))

(define-wrapper (baz [foo #:! a #:? b #:? [c 'cc]]
                     #:! d #:? [e 'e])
  #:call-wrapped call-foo
  (displayln e)
  (set! a (+ a d))
  (define-values (res-a res-b res-c) (call-foo))
  (displayln res-a)
  res-b)

expands in full racket to:

(define (foo #:a a #:b [b 3] #:c [c 2])
    (let ([b (if (eq? b no-value) 3 b)]
          [c (if (eq? c no-value) 2 c)])
      (values a b c)))

(define (bar #:a a #:b (b no-value) #:c (c 'cc))
  (let ([c (if (eq? c no-value) 'cc c)])
    (foo #:a a #:b b #:c c)))

(define (baz #:a a #:b (b no-value) #:c (c 'cc) #:d d #:e (e 'e))
  (let ([c (if (eq? c no-value) 'cc c)]
        [e (if (eq? e no-value) 'e e)])
    (let ((call-foo (λ () (foo #:a a #:b b #:c c))))
      (displayln e)
      (set! a (+ a d))
      (define-values (res-a res-b res-c) (call-foo))
      (displayln res-a)
      res-b)))

How does it look?

[Metaxal]

(also, that would be a nice use of immutable boxes :smiley: )

[rocketnia]

Looks wonderful to me! I think I'm fresh out of other suggestions. :)

[Metaxal]

Pushed to define2. Thanks for your help!