Providing a macro-function for non-ANSI special-operators

[digikar99]

CLHS apparantly requires implementations to provide a macro-function for non-ANSI special-operators http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm with the line:

The macro definition must be available for use by programs that understand only the standard Common Lisp special forms.

As of Version 1.12, macro-functions are absent for special operators like ccl:compiler-let ccl:nfunction. For some special operations like sb-kernel:the*, SBCL provides both a macro-function and implements it as a special-operator.

[phoe]

I would not take that sentence out of context. Let me quote the full paragraph:

It is possible for both macro-function http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm#macro-function and special-operator-p http://www.lispworks.com/documentation/lw71/CLHS/Body/f_specia.htm#special-operator-p to return /true/ http://www.lispworks.com/documentation/lw71/CLHS/Body/26_glo_t.htm#true of /symbol/. The /macro/ http://www.lispworks.com/documentation/lw71/CLHS/Body/26_glo_m.htm#macro definition must be available for use by programs that understand only the standard Common Lisp /special forms/ http://www.lispworks.com/documentation/lw71/CLHS/Body/26_glo_s.htm#special_form.

To me, this means that a symbol can name both a special operator and a macro, and that the programmer can expect such a situation. Then, if and only if that is the case, the macroexpansion must be expressed in terms of only standard Common Lisp special operators. Therefore, if a special operator does not name a macro, then there is, naturally, no macroexpansion of it.

Also, SBCL does not provide a macroexpansion for every custom special operator that it provides - see the following code run on SBCL 2.1.9.

CL-USER> (let (symbols)            (do-all-symbols (x symbols)              (when (special-operator-p x)                (pushnew x symbols)))            (let* ((predicate (lambda (x) (eq (find-package :common-lisp)                                              (symbol-package x))))                   (custom-specops (remove-if predicate symbols)))              (every #'macro-function custom-specops))) NIL

On 28.10.2021 18:29, Shubhamkar Ayare wrote:

CLHS apparantly requires implementations to provide a macro-function for non-ANSI special-operators http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm with the line:

The macro definition must be available for use by programs that
understand only the standard Common Lisp special forms.

As of |Version 1.12|, macro-functions are absent for special operators like |ccl:compiler-let ccl:nfunction|. For some special operations like |sb-kernel:the*|, SBCL provides both a macro-function and implements it as a special-operator.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Clozure/ccl/issues/398, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADSZHKUCA2GKQ4B5YPYUZY3UJF26HANCNFSM5G5KBAVA. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[digikar99]

Yes it is certainly true that SBCL has other symbols which are special-operators but do not have a macro-function. But I'm not sure about how to interpret the CLHS.

If both statements are considered, I do not see the second sentence adding any more information to the first. It trivially follows (unless I'm missing something!) from the first sentence that the macro-function is available if macro-function is true aka non-NIL. (But stating obvious things is okay!)

If only the second statement is considered, then implementations seem to be mandated to provide a macro-function for impl-specific special forms, even though the implementation itself may not use the macro-function. And going by this, SBCL too does fail to abide by it. I do not know its utility though. IIUC, the linked cl-form-types issue can be resolved by simply specializing block-type-walk-list-form on the appropriate symbol.

[phoe]

On 28.10.2021 23:39, Shubhamkar Ayare wrote:

If only the second statement is considered, then implementations seem to be mandated to provide a macro-function for impl-specific special forms, even though the implementation itself may not use the macro-function. This interpretation doesn't make much sense to me. A custom special operator would make no sense if an implementation was required to implement it in terms of a macro that must not, in turn, use custom special operators - because then custom special operators could, well, just be macros anyway, so why even call them specops?

[digikar99]

I think the requirement - if not optional - is only that the macro-function be provided. The macro-function itself may be implemented in terms of impl-specific special-operator; the mere provision helps it to be "use[d] by programs that understand only the standard Common Lisp special forms", am I missing something?

Also:

https://github.com/sbcl/sbcl/blob/7b64101fbba194fb6d5af1dd637defb1d26d7f3a/src/compiler/ir1-translators.lisp#L1027-L1034

;;; For the benefit of code-walkers we also add a macro-expansion. (Using INFO
;;; directly to get around safeguards for adding a macro-expansion for special
;;; operator.) Because :FUNCTION :KIND remains :SPECIAL-FORM, the compiler
;;; never uses the macro -- but manually calling its MACRO-FUNCTION or
;;; MACROEXPANDing TRULY-THE forms does.

So, this does form one use-case for why a macro-function might be provided for special-operators.

About the interpretation, posted on SBCL-Help mailing list as well: https://sourceforge.net/p/sbcl/mailman/sbcl-help/?viewmonth=202110&viewday=29.

[informatimago]

Le 29/10/2021 à 10:10, Shubhamkar Ayare a écrit :

I think the requirement - if not optional - is only that the macro-function be provided. The macro-function itself may be implemented in terms of impl-specific special-operator; the mere provision helps it to be "use[d] by programs that understand only the standard Common Lisp special forms", am I missing something?

No, this is correct. The point is to support code walkers (and other tools that need to "interpret" lisp code.

The semantics of standard special operators are specified by the standard; a code walker can therefore interpret them without difficulty.

The semantics of function calls are specified by the standard; a code walker can therefore interpret them without difficulty.

The semantics of macro calls are also specified by the standard, by calling the macro-function and substituting the macro call with the returned form, and repeating until we resolve down to only function calls or special operators.

But there remains the problem of non-standard special operators. For those, the implementation must provide an "equivalent" macro so that code walker can remove the non-standard special operator by expanding the macro, and resolving eventually only to function calls and standard special operators.

Note that this is not a strong constraint, because all macros, can, semantically, expand to a single function call, with parameters, including lambda expressions for the bindings.

For a code walker, the important semantic question is to determine for each argument to the macro, whether it's the name of a binding (and of what kind), to be used to augment the environment, or if it's an expression to be further evaluated (and in which environment).

For example, a dolist macro could expand to a function call such as:

(dolist (var list-expression result-expression) . body) --> (call-dolist list-expression (lambda (var) . body) (lambda (var) result-expression))

From this form, a code walker can determine that list-expression is an expression to be evaluated in the current environment, that var is a new binding for an environment where the body expressions will be evaluated, and that the result-expression is also an expression that may be evaluated in an environment where var is also defined.

Other kind of bindings (functions, blocks, etc) can be thus described expanding a form containing flet, block or other macros or special operators.

https://zenodo.org/record/3254669#.YXuw9C8Rpqs

The macro provided must eventually expand to a form without any implementation specific special operators. Only function calls and special operator.

This "eventually expand" is the expresion performed by macroexpand-all operators provided by various utilities including slime. (macroexpand only expands the toplevel form, not its subexpressions).

And yes, implementations defininig implementation specific special operators MUST provide such a macro. But since as explained, such a macro can expand to a call to an implementation-specific function, while it is sufficient for a code walker to interpret correctly the subexpressions/arguments to the macro/special operator, it may not give much information about the semantics of the operator. For example, call-dotimes would have the same signature (only list-expression would be an integer-expression), as would a (call-if test (lambda () then) (lambda () else)) and so on. Nothing indicates what will be done of the arguments to those implementation-specific functions.

So there is no excuse for an implementation, not to provide those macros for their implementation-specific special operators! That would be a clear case of non-conformity.

-- Pascal Bourguignon