Regressions in 2.0.8 compared to 0.9.6

[christoff-buerger]

Important note: The tests Ypsilon fails are passed by other R6RS Scheme systems like Chez Scheme, Larceny, Racket and Sagittarius Scheme. Hence, I think these are indeed Ypsilon compiler bugs.

System setup: Just in case this is important, I am using an old MacBook Air 2011 (x86, Intel). I build Ypsilon with LLVM/Clang 15.0.7.

I recently switched from Ypsilon 0.9.6-trunk/r506 to 2.0.8. The new version fails 6 tests of my Scheme R6RS project, whereas the old failed only a single (see https://github.com/christoff-buerger/racr/issues/88).

The kind of errors I now get with 2.0.8 are:

error in C: expected 1, but 0 argument given

backtrace:
  0  (C)
  ... racr/tests/rewrite-basics.scm:111
  1  (construct-reevaluation-tests)
  ... racr/tests/rewrite-basics.scm:12

and

error in append: expected proper list, but got #<unspecified>, as argument 1

irritants:
  (#<unspecified> ())

expanding:
  >  (transition: c ((D (token (eq? token Token)))) ((A 'Euro)))
  ... racr/tests/../examples/atomic-petrinets/examples/cookie-automaton.scm:33

and

error in syntax template: subforms have different size of matched input

irritants:
  (template: (:Transition 'name (list (:Arc 'input-place ...) ...) ...) ...))
  (subforms: (output-place ()) (to-produce ()) (input-place (A A)) (matching-condition (() ())) (variable (() ())) (name a))

expanding:
  >  (transition: a ((A) (A)) ())
  ... racr/tests/../examples/atomic-petrinets/examples/syntax-tests.scm:25
  *  (assert-exception condition? (petrinet: ((A)) (transition: a ((A) (A)) ())))
  ... racr/tests/../examples/atomic-petrinets/examples/syntax-tests.scm:23
  *  (assert-exception (petrinet: ((A)) (transition: a ((A) (A)) ())))
  ... racr/tests/../examples/atomic-petrinets/examples/syntax-tests.scm:23

The first error is for a normal function that edits graphs represented using R6RS records (hence, the set! operations on the fields encoding edges is used to change the graph structure), the others look like macros/syntax-forms expansion issues.

It is very hard for me to understand the Ypsilon code, and why it fails compared to the old version. But my Scheme library provides a convenient set of Bash scripts and tests which should make it easy for you to check and debug the errors yourself.

If you like to do that, please:

1. Have Ypsilon on your path, with the R6RS standard library/sitelib a direct subdirectory where your executable is.
2. Clone my git repository from https://github.com/christoff-buerger/racr
3. Run tests/execute.bash -s ypsilon -x; this will run all tests with the erroneous failing.
4. To run a single scheme program using my library you can also just use the deploying/deployment-scripts/execute.bash script.

[fujita-y]

Hello Christoff,

Thank you for reporting the issues! I've successfully reproduced the bug and started working on it.

Best, Yoshikatsu

[fujita-y]

Update: Found bug in syntax transformer in both syntax-rules and syntax-case.

(define-syntax bad
  (syntax-rules ()
    ((_ ((x ...) ...) (y ...))
     (quote ((y ((x ... ...))) ...)))))

(bad ((x1 x2 x3)) (y1 y2)) ;=> internal error

(define-syntax bad
  (lambda (x)
    (syntax-case x ()
      ((_ ((x ...) ...) (y ...))
       #`(quote ((y ((x ... ...))) ...))))))

(bad ((x1 x2 x3)) (y1 y2)) ;=> internal error

While simple use of '... ...' is ok:

(define-syntax ok
  (syntax-rules ()
    ((_ (x ...) ...)
     (quote (x ... ...)))))

(ok (a b c) (d e) (f)) ;=> '(a b c d e f)

[fujita-y]

Update: Macro expander fix is done (branch: bugfix). I found that stdlib/core/optimizer.scm have problem. Temporary disable optimization fixed another issue. Continue working on fix optimizer. Thanks!

[christoff-buerger]

Dear Yoshikatsu,

Thank you very much for looking into it! This was very fast :)

I just tested your bugfix branch, and all but two of the errors are gone. The remaining ones are:

error in C: expected 1, but 0 argument given

backtrace:
  0  (C)
  ... /Users/cbuerger/Documents/development/racr/tests/rewrite-basics.scm:111
  1  (construct-reevaluation-tests)
  ... /Users/cbuerger/Documents/development/racr/tests/rewrite-basics.scm:12

and

error: Assertion Failed!

backtrace:
  0  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  1  (f pos (ast-child pos n))
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:1470
  2  (when (<= pos ub) (f pos (ast-child pos n)) (set! pos (+ pos 1)) (loop))
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:1469
  3  (|.dynamic-wind| |.L~249| (lambda () (letrec* ((loop (lambda () (if (|.<=| pos ...) ...) ...) ...) ...) ...) ...) ...)
  4  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  5  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  6  (siple-interpret program)
  ... /Users/cbuerger/Documents/development/racr/tests/../examples/siple/execute.scm:19

I guess the first one -- the error in C: expected 1, but 0 argument given error -- is the optimization issue you mention. This one is new in 2.0.8 compared to 0.9.6.

The second error -- the error: Assertion Failed! error -- was already present in 0.9.6.

I am glad to help; just tell me if you need more details.

[fujita-y]

Update:

Successfully narrow downed the issue as follows :D

(define Y (lambda() #t))

(define run-tests
  (lambda ()
      (let* ((X Y)
             (S (lambda () X))
             (A (lambda () (list 1 (S))))
             (C (lambda () (list 2 (A))))
             (D2 (lambda () (list 1 (C)))))
        (list (D2) (D2)))))

(run-tests)

error in C: expected 1, but 0 argument given

backtrace:
  0  (C)
  ... /dev/stdin:7
  1  (run-tests)
  ... /dev/stdin:1

And seeing the bug in lambda lifting :|

> (format #t "~y" 
    (macro-expand '(define run-tests
      (lambda ()
        (let* ((X Y)
               (S (lambda () X))
               (A (lambda () (list 1 (S))))
               (C (lambda () (list 2 (A))))
               (D2 (lambda () (list 1 (C)))))
          (list (D2) (D2)))))))
(begin
  (define |C`59*| (lambda (|.L~32|) (list 2 (list 1 |.L~32|))))
  (define run-tests (lambda () (list (list 1 (list 2 (list 1 Y))) (list 1 (|C`59*|))))))

[fujita-y]

Hi Christoff,

Issue should be fixed in master branch. Please check!

Best, Yoshikatsu

[christoff-buerger]

Issue should be fixed in master branch. Please check!

Looks good. I can confirm the error

error in C: expected 1, but 0 argument given

backtrace:
  0  (C)
  ... /Users/cbuerger/Documents/development/racr/tests/rewrite-basics.scm:111
  1  (construct-reevaluation-tests)
  ... /Users/cbuerger/Documents/development/racr/tests/rewrite-basics.scm:12

is gone.

Thanks a lot!

The following error for the /examples/siple/examples/correct/closures.siple example in my racr repository is still there though:

error: Assertion Failed!

backtrace:
  0  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  1  (f pos (ast-child pos n))
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:1470
  2  (when (<= pos ub) (f pos (ast-child pos n)) (set! pos (+ pos 1)) (loop))
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:1469
  3  (|.dynamic-wind| |.L~437| (lambda () (letrec* ((loop (lambda () (if (|.<=| pos ...) ...) ...) ...) ...) ...) ...) ...)
  4  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  5  (apply value node args)
  ... /Users/cbuerger/Documents/development/racr/racr/binaries/ypsilon/racr/core.scm:532
  6  (siple-interpret program)
  ... /Users/cbuerger/Documents/development/racr/tests/../examples/siple/execute.scm:19

Above failed assertion is not a regression compared to ypsilon 0.9.6. The assertion failes since ever for ypsilon. But it passes in other R6RS Scheme systems.

@fujita-y: Shall I open this as a separate issue since it is not a regression compared to version 0.9.6?

[fujita-y]

Yes, please open separate issue. Thanks!