support macro hiding in racket-stepper mode?

[bremner]

Is it possible to support something like DrRacket's macro hiding options? Even just "standard" would be a big help.

[greghendershott]

racket-expand-file varies depending on whether you supply a prefix (e.g. C-u). With a prefix, it shows expansion steps for racket/base. Otherwise it does not, which I believe is close to "standard hiding" in DrR -- but maybe that differs in some way that's important to you, and that's the feature request here? OR, maybe there's a bug even for the intended status quo?

Do you have some small example program, and examples of expansion steps you would and would not want to see?

[greghendershott]

From a quick experiment, it looks to me like the racket/base hiding is still working.

Given:

#lang racket/base
(define (hello _)
  (let ([x 1])
    x))

M-x racket-expand-file produces just no steps.

Whereas with C-u M-x racket-expand-file, there are many:

Original
(module issue-696 racket/base
  (#%module-begin (define (hello _) (let ((x 1)) x))))

1: Macro transformation
@@ -1,2 +1,8 @@
 (module issue-696 racket/base
-  (#%module-begin (define (hello _) (let ((x 1)) x))))
+  (printing:module-begin:1
+   (module:1
+    configure-runtime:1
+    '#%kernel:1
+    (#%require:1 racket/runtime-config:1)
+    (configure:1 #f))
+   (define (hello _) (let ((x 1)) x))))

2: Macro transformation
@@ -1,8 +1,12 @@
 (module issue-696 racket/base
-  (printing:module-begin:1
-   (module:1
-    configure-runtime:1
-    '#%kernel:1
-    (#%require:1 racket/runtime-config:1)
-    (configure:1 #f))
-   (define (hello _) (let ((x 1)) x))))
+  (#%module-begin:2
+   (do-wrapping-module-begin:2
+    print-result:2
+    (module:1
+     configure-runtime:1
+     '#%kernel:1
+     (#%require:1 racket/runtime-config:1)
+     (configure:1 #f)))
+   (do-wrapping-module-begin:2
+    print-result:2
+    (define (hello _) (let ((x 1)) x)))))

3: Macro transformation
@@ -1,7 +1,6 @@
 (module issue-696 racket/base
   (#%module-begin:2
-   (do-wrapping-module-begin:2
-    print-result:2
+   (begin:3
     (module:1
      configure-runtime:1
      '#%kernel:1

4: Splice module-level begin
@@ -1,11 +1,10 @@
 (module issue-696 racket/base
   (#%module-begin:2
-   (begin:3
-    (module:1
-     configure-runtime:1
-     '#%kernel:1
-     (#%require:1 racket/runtime-config:1)
-     (configure:1 #f)))
+   (module:1
+    configure-runtime:1
+    '#%kernel:1
+    (#%require:1 racket/runtime-config:1)
+    (configure:1 #f))
    (do-wrapping-module-begin:2
     print-result:2
     (define (hello _) (let ((x 1)) x)))))

5: Add explicit #%module-begin
@@ -3,8 +3,7 @@
    (module:1
     configure-runtime:1
     '#%kernel:1
-    (#%require:1 racket/runtime-config:1)
-    (configure:1 #f))
+    (#%module-begin:1 (#%require:1 racket/runtime-config:1) (configure:1 #f)))
    (do-wrapping-module-begin:2
     print-result:2
     (define (hello _) (let ((x 1)) x)))))

6: Add explicit #%app
@@ -3,7 +3,9 @@
    (module:1
     configure-runtime:1
     '#%kernel:1
-    (#%module-begin:1 (#%require:1 racket/runtime-config:1) (configure:1 #f)))
+    (#%module-begin:1
+     (#%require:1 racket/runtime-config:1)
+     (#%app:1 configure:1 #f)))
    (do-wrapping-module-begin:2
     print-result:2
     (define (hello _) (let ((x 1)) x)))))

7: Add explicit #%datum
@@ -5,7 +5,7 @@
     '#%kernel:1
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
-     (#%app:1 configure:1 #f)))
+     (#%app:1 configure:1 (#%datum:1 . #f))))
    (do-wrapping-module-begin:2
     print-result:2
     (define (hello _) (let ((x 1)) x)))))

8: Macro transformation
@@ -5,7 +5,7 @@
     '#%kernel:1
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
-     (#%app:1 configure:1 (#%datum:1 . #f))))
+     (#%app:1 configure:1 '#f)))
    (do-wrapping-module-begin:2
     print-result:2
     (define (hello _) (let ((x 1)) x)))))

9: Macro transformation
@@ -1 +1 @@
-(define (hello _) (let ((x 1)) x))
+(define:4 hello (lambda:4 (_) (let ((x 1)) x)))

10: Macro transformation
@@ -1 +1 @@
-(define:4 hello (lambda:4 (_) (let ((x 1)) x)))
+(define-values:5 (hello) (lambda:4 (_) (let ((x 1)) x)))

11: Macro transformation
@@ -6,6 +6,4 @@
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
      (#%app:1 configure:1 '#f)))
-   (do-wrapping-module-begin:2
-    print-result:2
-    (define (hello _) (let ((x 1)) x)))))
+   (begin:6 (define-values:5 (hello) (lambda:4 (_) (let ((x 1)) x))))))

12: Splice module-level begin
@@ -6,4 +6,4 @@
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
      (#%app:1 configure:1 '#f)))
-   (begin:6 (define-values:5 (hello) (lambda:4 (_) (let ((x 1)) x))))))
+   (define-values:5 (hello) (lambda:4 (_) (let ((x 1)) x)))))

13: Macro transformation
@@ -6,4 +6,4 @@
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
      (#%app:1 configure:1 '#f)))
-   (define-values:5 (hello) (lambda:4 (_) (let ((x 1)) x)))))
+   (define-values:5 (hello) (lambda:4 (_) (let-values:7 (((x) 1)) x)))))

14: Add explicit #%datum
@@ -6,4 +6,6 @@
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
      (#%app:1 configure:1 '#f)))
-   (define-values:5 (hello) (lambda:4 (_) (let-values:7 (((x) 1)) x)))))
+   (define-values:5
+    (hello)
+    (lambda:4 (_) (let-values:7 (((x) (#%datum . 1))) x)))))

15: Macro transformation
@@ -6,6 +6,4 @@
     (#%module-begin:1
      (#%require:1 racket/runtime-config:1)
      (#%app:1 configure:1 '#f)))
-   (define-values:5
-    (hello)
-    (lambda:4 (_) (let-values:7 (((x) (#%datum . 1))) x)))))
+   (define-values:5 (hello) (lambda:4 (_) (let-values:7 (((x) '1)) x)))))

Final
(module issue-696 racket/base
  (#%module-begin:2
   (module:1
    configure-runtime:1
    '#%kernel:1
    (#%module-begin:1
     (#%require:1 racket/runtime-config:1)
     (#%app:1 configure:1 '#f)))
   (define-values:5 (hello) (lambda:4 (_) (let-values:7 (((x) '1)) x)))))

So I think it's working as intended; I'll learn more about DrR "standard hiding", and if you have a tangible example you care about personally, that might also be useful for me to test.

[greghendershott]

After investigating: It looks like DrRacket "standard" hiding includes racket/base, and also a few more things. Furthermore DrRacket in general supports a variety of rules; see macro-debugger/model/hiding-policies.

I have some commits to support nearly all of this: "disabled", "standard" (as in DrR), and an arbitrary custom sexpr for which nearly all the rules should work (except (free=? identifier), which needs a live syntax object that we can't supply from Emacs).

---

I can merge all this but I would still love to hear back from you -- with an example or two where you wanted hiding that was happening with DrR "standard" but not in status quo Racket Mode. That way I can do some more testing and make sure the work I've just done will actually be of value for you.

[bremner]

Greg Hendershott @.***> writes:

After investigating: It looks like DrRacket "standard" hiding includes racket/base, and also a few more things. Furthermore DrRacket in general supports a variety of rules; see macro-debugger/model/hiding-policies.

I have some commits to support nearly all of this: "disabled", "standard" (as in DrR), and an arbitrary custom sexpr for which nearly all the rules should work (except (free=? identifier), which needs a live syntax object that we can't supply from Emacs).

---

Hi Greg;

Thanks for looking into this.

At the end of this message is what I think is a representative example. In Dr Racket with "standard" hiding I get 4 steps, while in racket-mode there are 6. It seems to be related to "hide-library-syntax" (option in DrRacket custom macro hiding), but I don't know if that if that is the whole story. Let me know if you need/want more examples; the particular unwanted expansions (related to the required macros, I guess), seem consistent across the examples I looked at.

#lang racket
(require [only-in plait test test/exn error print-only-errors])

(define-syntax my-let*
  (syntax-rules ()
    [(my-let* () body) body]
    [(my-let* ([v0 e0] [v1 e1] ...) body)
     (let ([v0 e0])
       (my-let* ([v1 e1] ...) body))]))

(test (my-let* ([x 1]
                [y (+ x 2)]
                [x (+ y 3)])
         x)
      6)

[greghendershott]

Thanks!

With that example:

• With my changes to racket-expand-file described previously, I do get 4 steps:

Original
(module issue-697 racket
  (#%module-begin
   (require (only-in plait test test/exn error print-only-errors))
   (define-syntax my-let*
     (syntax-rules ()
       ((my-let* () body) body)
       ((my-let* ((v0 e0) (v1 e1) ...) body)
        (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
   (test (my-let* ((x 1) (y (+ x 2)) (x (+ y 3))) x) 6)))

1: Macro transformation
@@ -6,4 +6,4 @@
        ((my-let* () body) body)
        ((my-let* ((v0 e0) (v1 e1) ...) body)
         (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
-   (test (my-let* ((x 1) (y (+ x 2)) (x (+ y 3))) x) 6)))
+   (test (let:1 ((x 1)) (my-let*:1 ((y (+ x 2)) (x (+ y 3))) x)) 6)))

2: Macro transformation
@@ -6,4 +6,4 @@
        ((my-let* () body) body)
        ((my-let* ((v0 e0) (v1 e1) ...) body)
         (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
-   (test (let:1 ((x 1)) (my-let*:1 ((y (+ x 2)) (x (+ y 3))) x)) 6)))
+   (test (let:1 ((x 1)) (let:2 ((y (+ x 2))) (my-let*:2 ((x (+ y 3))) x))) 6)))

3: Macro transformation
@@ -6,4 +6,8 @@
        ((my-let* () body) body)
        ((my-let* ((v0 e0) (v1 e1) ...) body)
         (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
-   (test (let:1 ((x 1)) (let:2 ((y (+ x 2))) (my-let*:2 ((x (+ y 3))) x))) 6)))
+   (test
+    (let:1
+     ((x 1))
+     (let:2 ((y (+ x 2))) (let:3 ((x (+ y 3))) (my-let*:3 () x))))
+    6)))

4: Macro transformation
@@ -6,8 +6,4 @@
        ((my-let* () body) body)
        ((my-let* ((v0 e0) (v1 e1) ...) body)
         (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
-   (test
-    (let:1
-     ((x 1))
-     (let:2 ((y (+ x 2))) (let:3 ((x (+ y 3))) (my-let*:3 () x))))
-    6)))
+   (test (let:1 ((x 1)) (let:2 ((y (+ x 2))) (let:3 ((x (+ y 3))) x))) 6)))

Final
(module issue-697 racket
  (#%module-begin
   (require (only-in plait test test/exn error print-only-errors))
   (define-syntax my-let*
     (syntax-rules ()
       ((my-let* () body) body)
       ((my-let* ((v0 e0) (v1 e1) ...) body)
        (let ((v0 e0)) (my-let* ((v1 e1) ...) body)))))
   (test (let:1 ((x 1)) (let:2 ((y (+ x 2))) (let:3 ((x (+ y 3))) x))) 6)))

• When I try in DrR, I get 4 steps labeled "Macro transformation" -- plus a few more labeled "resolve variable (remove extra scopes)". Huh.
  • Do you see these in DrR, too?
  • I don't understand why I see them in DrR but not in Racket Mode via macro-debugger/stepper-text.

TL;DR I think it's good to merge... but the discrepancy makes me worry it's not quite right and I'm overlooking something??

[bremner]

Greg Hendershott @.***> writes:

• With my changes to racket-expand-file described previously, I do get 4 steps:

[snipped, but looks good to me]

• When I try in DrR, I get 4 steps labeled "Macro transformation" -- plus a few more labeled "resolve variable (remove extra scopes)". Huh.
  • Do you see these in DrR, too?

I only see those if I enable Stepper -> Extra Options -> Include Renaming steps.

• I don't understand why I see them in DrR but not in Racket Mode via macro-debugger/text-stepper.

Do you maybe have that extra option enabled?

TL;DR I think it's good to merge... but the discrepancy makes me suspect it's not quite right and I'm overlooking something??

[greghendershott]

I only see those if I enable Stepper -> Extra Options -> Include Renaming steps.

Oh! Thanks; I didn't even realize that option existed. Yep, it was enabled.

In that case I think this might be ready. I'll give it one more look before merging, probably today...

[greghendershott]

Update: I'm taking a little more time to refine the UX around the racket-expand-hiding customization variable. Might commit today.

[greghendershott]

Merged. There's a new customization variable, racket-expand-hiding. For what you originally requested, you can simply set this to 'standard; please let me know if you have any question or problems.

It took me a few more days to finish this because I made this also able to reflect the "Custom" checkboxes and rules in the DrRacket GUI. The ergonomics of the customize-variable TUI isn't amazing, but, it works. Also the defcustom :type serves as a kind of grammar for supplying the values directly as an Emacs Lisp s-expression, via setq or the new setopt.

[bremner]

Greg Hendershott @.***> writes:

Merged. There's a new customization variable, racket-expand-hiding. For what you originally requested, you can simply set this to 'standard; please let me know if you have any question or problems.

At least for my simple examples from class, it works great. Thanks a lot for this improvement.

David