special-form inside `block` errors on CCL

[digikar99]

An example:

CL-USER> (cl-form-types:nth-form-type
          `(block nil
             (ccl:compiler-let ()))
          nil)

Encountered an unknown special operator COMPILER-LET, with operands:
  (NIL)
   [Condition of type CL-FORM-TYPES:UNKNOWN-SPECIAL-OPERATOR]

Restarts:
 0: [RETURN-DEFAULT-TYPE] #<RESTART CL-FORM-TYPES:RETURN-DEFAULT-TYPE #x7FC330D860BD>
 1: [RETRY] Retry SLIME REPL evaluation request.
 2: [*ABORT] Return to SLIME's top level.
 3: [ABORT-BREAK] Reset this thread
 4: [ABORT] Kill this thread

Backtrace:
  0: (#<STANDARD-METHOD CL-FORM-TYPES::BLOCK-TYPE-WALK-LIST-FORM (T T T)> COMPILER-LET (NIL) NIL)
  1: (NIL #<Unknown Arguments>)
  2: ((CCL::TRACED CL-FORM-TYPES::BLOCK-TYPE-WALK-LIST-FORM) COMPILER-LET (NIL) NIL)
  3: (CL-FORM-TYPES::WALK-FORM% (COMPILER-LET ()) NIL)
  4: ((:INTERNAL ALEXANDRIA:RCURRY) (COMPILER-LET ()))
  5: (MAP NIL #<COMPILED-LEXICAL-CLOSURE (:INTERNAL ALEXANDRIA:RCURRY) #x3020033C847F> ((COMPILER-LET ())))
  6: ((:INTERNAL (CL-FORM-TYPES::WALK-LIST-FORM (T T T))) (PROGN (COMPILER-LET ())) NIL)
  7: (NIL #<Unknown Arguments>)
  8: ((:INTERNAL CL-FORM-TYPES::WALK-FORM%) (PROGN (COMPILER-LET ())) NIL)
  9: (CL-FORM-TYPES::WALK-FORM #<Compiled-function (:INTERNAL CL-FORM-TYPES::WALK CL-FORM-TYPES::BLOCK-TYPE-WALK-FORM) (Non-Global)  #x3020012DD39F> (PROGN (COMPILER-LET ())) NIL :RESULT-TYPE NIL)
 10: (CL-FORM-TYPES::EXTRACT-RETURN-FROM-TYPES NIL (PROGN (COMPILER-LET ())) NIL)
 11: (#<STANDARD-METHOD CL-FORM-TYPES::SPECIAL-FORM-TYPE ((EQL BLOCK) T T)> BLOCK (NIL (COMPILER-LET ())) NIL)
 12: (CCL::%CALL-NEXT-METHOD (NIL #<STANDARD-METHOD CL-FORM-TYPES::SPECIAL-FORM-TYPE ((EQL BLOCK) T T)> . 17559539630812))

I have a small doubt about interpreting "The macro definition must be available for use by programs that understand only the standard Common Lisp special forms." on CLHS: http://www.lispworks.com/documentation/lw71/CLHS/Body/f_macro_.htm

Is that indicating that the bug is on CCL side and not cl-form-types - or is that inconsequential and the bug is on cl-form-types?

[alex-gutev]

The bug is technically on CCL's side, they don't provide a macro function for compiler-let, though I don't think it's even possible to implement it as a macro. Many implementations, I think even SBCL, ignore this point of the spec at least for some of their implementation-specific special forms.

In cl-form-types it is first checked whether an expression operator has a macro function attached to it, if so the macro function is invoked. If the operator does not have a macro-function and is not one of the standard Common Lisp special operators, or a known implementation specific special operator, the unknown-special-operator condition is signaled. The return-default-type restart allows you to treat the form as though it always returns a value of a specific type (default T). However in this case, the type specifier given to the return-default-type should be returned as the form type for the entire block form, not just the compiler-let special form. If this is not the case then there is a bug in cl-form-types. This is due to the fact that the return type of a block form is affected by go forms nested inside the special forms, which cannot be analyzed, hence simply the default type of t should be used.

Edit: Most likely invoking the restart will only affect the result of form-type on the special form, thus giving you a potentially erroneous type for the block form.

[digikar99]

Thanks for confirming! Created an issue: https://github.com/Clozure/ccl/issues/398

[digikar99]

And yup, SBCL does not have a macro-function for sb-cltl2:compiler-let.

. . .

Turns out http://www.lispworks.com/documentation/lw50/CLHS/Issues/iss066_w.htm mentions about a possible way to implement compiler-let as a macro, in the presence of a macroexpand-all:

      (DEFMACRO COMPILER-LET (BINDINGS &BODY FORMS &ENVIRONMENT ENV)
        (SETQ BINDINGS ;; Assure no non-atom bindings
          (MAPCAR #'(LAMBDA (BINDING) 
                  (IF (ATOM BINDING) (LIST BINDING) BINDING))
              BINDINGS))
        (PROGV (MAPCAR #'CAR BINDINGS)
           (MAPCAR #'(LAMBDA (BINDING) (EVAL (CADR BINDING))) BINDINGS)
      (SYSTEM::MACROEXPAND-ALL `(PROGN ,@FORMS) ENV)))

However, I don't think I get the detail about semantic prepass, and its implications.

[alex-gutev]

I don't know if anyone on CCL's or SBCL's team will actually consider it as a bug. Both, and most other implementations, are filled with nonstandard special forms for which there are no macro-functions, despite the fact that theoretically the spec forbids it. But anyway its a good thing you pointed it out. I think this issue should be closed now?

[digikar99]

Yup, I'll see where the discussion heads.

And potentially create a PR with block-type-walk-list-form specializing on ccl:compiler-let in case that's possible - the definition above does seem alright at least for a code-walker though?