search

clasp-developers / clasp

clasp Common Lisp environment
https://clasp-developers.github.io/
2.57k stars 145 forks source link

Cross-compiled sbcl dies in make-target-2.sh #945

Open kpoeck opened 4 years ago

kpoeck commented 4 years ago

With the following changes clasp can crosscompile sbcl:

kpoeck commented 4 years ago

With all the pre-conditions as in the first comment, the following happens at the beginning of make-target-2.sh:

//entering make-target-2.sh
//doing warm init - compilation phase
This is SBCL 2.0.2.122.feature-fix-undefined-defstruct-behavior.1-4d78b991b-dirty, an implementation of ANSI Common Lisp.
More information about SBCL is available at <http://www.sbcl.org/>.

SBCL is free software, provided as is, with absolutely no warranty.
It is mostly in the public domain; some portions are provided under
BSD-style licenses.  See the CREDITS and COPYING files in the
distribution for more information.
Initial page table:
Gen  Boxed   Code    Raw  LgBox LgCode  LgRaw  Pin       Alloc     Waste        Trig      WP GCs Mem-age
 6     735      0      0      0      0      0    0    24052752     31728     2000000     735   0  0.0000
           Total bytes allocated    =      24052752
           Dynamic-space-size bytes =    1073741824
/entering !COLD-INIT
Internal error #11 "Object is of the wrong type." at 0x21ba85c1
    SC: 2, Offset: 0    0
    SC: 3, Offset: 11   $1=   0x1000105bf7: list pointer
fatal error encountered in SBCL pid 472(tid 0x1567f5c0):
internal error too early in init, can't recover

Welcome to LDB, a low-level debugger for the Lisp runtime environment.
ldb> backtrace
Backtrace:
   0: (FLET SB-IMPL::NEXT-FIELD :IN SB-IMPL::UNPACKIFY-INFOS), pc = 0x21ba85c1, fp = 0xe4d5c10
   1: SB-IMPL::UNPACKIFY-INFOS, pc = 0x21ba84ac, fp = 0xe4d5ca0
   2: SB-IMPL::PACKED-INFO-REMOVE, pc = 0x21ba99ef, fp = 0xe4d5d38
   3: (FLET SB-IMPL::CLEAR-SIMPLE :IN SB-INT::CLEAR-INFO-VALUES), pc = 0x21bae535, fp = 0xe4d5d88
   4: SB-INT::UPDATE-SYMBOL-INFO, pc = 0x21d026d2, fp = 0xe4d5dc0
   5: SB-INT::CLEAR-INFO-VALUES, pc = 0x21bae283, fp = 0xe4d5e48
   6: (SETF SYMBOL-FUNCTION), pc = 0x21d0232f, fp = 0xe4d5ec0
   7: SB-KERNEL::!COLD-INIT, pc = 0x227f238e, fp = 0xe4d5fc8
kpoeck commented 4 years ago

Following advise of karlosz, I added tracefiles for:

("src/code/cold-init" :not-host :trace-file)
("src/compiler/info-vector" :c-headers :trace-file)
("src/compiler/globaldb" :c-headers :trace-file)
kpoeck commented 4 years ago

Self compiled with sbcl this give the following output: cold-init.trace.sbcl.txt globaldb.trace.sbcl.txt info-vector.trace.sbcl.txt

Cross compiled with clasp it gives: cold-init.trace.clasp.txt globaldb.trace.clasp.txt info-vector.trace.clasp.txt

kpoeck commented 4 years ago

Obviosly there is some issue in clasps printers, snce symbols are printed differently.

<LABEL 1> vs #<SB-ASSEM:LABEL 1>

<COMPONENT .. vs #<SB-C:COMPONENT..

but also real differences like

    7: CALL-NAMED t4[RSP] :NORMAL t7[RDX] :NORMAL
                  {(#<TN t7[RDX] :NORMAL>) 1 FIND-UNDELETED-PACKAGE-OR-LOSE 1 NIL}
                  => t8[RDX] :NORMAL

vs
    7: CALL-NAMED t4[RSP] :NORMAL t7[RDX] :NORMAL
                  {# 1 FIND-UNDELETED-PACKAGE-OR-LOSE 1 NIL}
                  => t8[RDX] :NORMAL

the second w/o the TN

kpoeck commented 4 years ago

the output of context after the error:

ldb> context
There are 1 interrupt contexts.
printing context 0
RAX:          0x00000000: even fixnum: 0
RCX:          0x00000004: pad1: 2
RDX:    $2=   0x10016a4b6f: other pointer
RBX:          0x00000004: pad1: 2
RSP:          0x0e4d5b90: even fixnum: 119975368
RBP:          0x0e4d5c10: even fixnum: 119975432
RSI:          0x0e4d5c10: even fixnum: 119975432
RDI:    $3=   0x0e4d5cbf: other pointer
R8:       0xfffffffffffffffe: pad5: -1
R9:       0x0e4d5ca0: even fixnum: 119975504
R10:    $2=   0x10016a4b6f: other pointer
R11:    $4=   0x2033a93f: other pointer
R12:    $2=   0x10016a4b6f: other pointer
R13:          0x0e6d8000: even fixnum: 121028608
R14:          0x0e4d5ca0: even fixnum: 119975504
R15:          0x0000007e: pad5: 63
PC:       0x21ba85c1
karlosz commented 4 years ago

Potentially worrying difference: The label numbers differ sometimes


 ;;;; IR2 component: "top level form"

 entries:
-    L156: (SB-C::TOP-LEVEL-FORM NIL)
+    L155: (SB-C::TOP-LEVEL-FORM NIL)

 TNs: 9 local, 2 temps, 5 constant, 2 env, 1 comp, 2 global.
@@ -3481,11 +3534,11 @@
karlosz commented 4 years ago

This form may not be executing properly. It is what is controlling the label and continuation numbers, and should behave the same under any cross compilation host This is in compiler/debug.lisp.


;;; IDs so that we can get a handle on anonymous objects given a
;;; printout.
(macrolet ((def (array-getter fto ffrom type) ; "to" = to number/id, resp. "from"
             (let ((xform (if (eq array-getter 'objmap-id-to-ir2block)
                              'id->string
                              'identity)))
               `(progn
                  (export '(,fto ,ffrom))
                  (defun ,fto (x)
                    (declare (type ,type x))
                    (let* ((map *compilation*)
                           (ht (objmap-obj-to-id map)))
                      (or (gethash x ht)
                          (let ((array (,array-getter map)))
                            (when (null array)
                              (setf array (make-array 20 :fill-pointer 0
                                                      :adjustable t)
                                    (,array-getter map) array))
                            (setf (gethash x ht)
                                  (,xform (1+ (vector-push-extend x array))))))))
                  (defun ,ffrom (num)
                    (let ((array (,array-getter *compilation*)))
                      ,@(when (eq array-getter 'objmap-id-to-ir2block)
                          '((setq num (string->id num))))
                      ;; numbers start from 1, not 0
                      (and (< (1- num) (length array)) (aref array (1- num)))))))))
  (def objmap-id-to-node node-id id-node node)
  (def objmap-id-to-comp component-id id-component component)
  (def objmap-id-to-leaf leaf-id id-leaf leaf)
  (def objmap-id-to-cont %cont-num num-cont (or ctran lvar))
  (def objmap-id-to-ir2block ir2-block-id id-ir2-block ir2-block)
  (def objmap-id-to-tn tn-id id-tn tn)
  (def objmap-id-to-label label-id id-label label))
kpoeck commented 4 years ago

Update, with newest sbcl, there is another challenge for x-compiling sbcl.

I needed to change force-delayed-defbangstructs.lisp to the followng, so that it works:

;;;; Once all the cross-compiler DEFSTRUCT machinery has been set up,
;;;; we can feed the stored DEF!STRUCT argument lists to it. (This
;;;; goes in its own source file, instead of in the same file as the
;;;; DEFSTRUCT machinery, because it's tidier and more maintainable
;;;; than adding EVAL-WHEN :COMPILE wrappers to anything that it might
;;;; need.)

;;;; This software is part of the SBCL system. See the README file for
;;;; more information.
;;;;
;;;; This software is derived from the CMU CL system, which was
;;;; written at Carnegie Mellon University and released into the
;;;; public domain. The software is in the public domain and is
;;;; provided with absolutely no warranty. See the COPYING and CREDITS
;;;; files for more information.

(in-package "SB-KERNEL")
#|
(macrolet ((do-delayed-structs ()
             `(progn
                ,@(mapcan (lambda (x)
                            (list `(in-package ,(delayed-defstruct-package x))
                                  `(sb-xc:defstruct ,@(delayed-defstruct-args x))))
                          (reverse *delayed-defstructs*)))))
  (do-delayed-structs)
  #+no (makunbound '*delayed-defstructs*)
  )
|#

(mapcar #'(lambda(x)
            (eval
             (list
              'progn
              `(in-package ,(delayed-defstruct-package x))
              `(sb-xc:defstruct ,@(delayed-defstruct-args x)))))
        (reverse *delayed-defstructs*))