Blocker on gentoo-init due to sage ecls

[gagern]

Currently my world update fails because dev-lisp/gentoo-init is included as a blocker of dev-lisp/asdf-3.0.1. It seems to me that gentoo-init is being phased out, and apparently the only ebuild depending on it is dev-lisp/ecls-12.12.1 from the sage overlay. There is a more recent ecls-12.12.1-r4 in the main portage tree, which depends on asdf instead. But sci-mathematics/sage-5.9 depends on =dev-lisp/ecls-12.12.1, so we can't use that.

Please provide a dependency chain which avoids this fixed dep on gentoo-init, perhaps by providing an updated ecls ebuild in the overlay, or by changing the dep in the sage ebuild to ~dev-lisp/ecls-12.12.1.

[kiwifb]

Urgh.... the maxima ecl library used by sage won't build with ecls-12.12.1-r4. That's why it is currently pined down to that version. See http://thread.gmane.org/gmane.linux.gentoo.science/1953 So we have to fix this before moving on.

[gagern]

So the goal is making sage work with the latest maxima ebuild from the portage tree, possibly tweaking that ebuild where required, right? I must confess that I haven't found the single relevant problem description in that thread you linked, if there is one. I see a lot of various concerns, most of which seem to be resolved by now.

[Coacher]

@gagern Hi. The issue you've described only happens with asdf-3.0.1 which is ~arch keyworded now. There is no such issue with other versions of asdf. Do you really need that latest version or your whole system is ~arch?

[gagern]

@Coacher: My system is ~amd64 throughout. I'm not sure why portage reported a block on this, instead of the more common message about slot conflicts which I very regularly see due to specific sage deps.

[kiwifb]

OK here is the problem: http://article.gmane.org/gmane.linux.gentoo.science/1958

[kiwifb]

I am making some progress on the issue but I am at the stage where I need a specialist or serious doc on asdf-bundle. But we'll get there.

[gagern]

I got sage compiled with maxima 5.30.0 from portage, without further modifications to it or to sage except for the fact that I had to patch the maxima texinfo docs (bug 466730) and had invoked the sage ebuild directly, thus suppressing portage's dependency resolution. I'm not sure whether the fact that it did compile is a good indication that it will work, though. But it does look better than the error message from the mail you referenced.

[kiwifb]

Well, Andrey commented out the part building the maxima ecl library from the maxima ebuild so it won't appear. If you run sage -t /usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py you may find you have some issue.

[gagern]

True enough. ☹

[kiwifb]

I have an issue with the way ecls 12.12.1-r4 is built solved (double wammy of asdf-bundle.lisp - incompatible with each other) but then I am faced with the fact that asdf has changed enough that the old procedure to produce the maxima ecl library is no longer working. And I don't currently know how to make it work and the doc is unhelpful.

[gagern]

With asdf 3.0.1, the commented-out comments in the maxima ebuild will create a maxima.system.fasb without complaining. Does this help in any way?

[kiwifb]

Well, I had Andrey make some changes that would help but I have never managed to build maxima.system.fasb when I tried manually. So yeah it would help!

[gagern]

Sorry, this way dev-lisp/ecls-12.12.1 from the sage overlay, not the dev-lisp/ecls-12.12.1-r5 I had in mind. After installing the latter again, I get Component :MAXIMA not found. which is something with which I can work, I guess. Will try to see if I can figure something out, although I have very little experience with common lisp.

[kiwifb]

The way asdf-bundle works has changed considerably in asdf-3 I am of a mind to ask on their mailing list how we should migrate the procedure. But if you can work it out t hat would be great.

[gagern]

After spending several hours on this, I'm giving up, at least for today. Give the mailing list a go if you feel like it.

Findings so far: the removal of asdf-bundle.lisp from ecls the way our Gentoo patch for ecls-12.12.1-r5 does it is the critical point. With builtin asdf-bundle, things seem to work, even alongside asdf-3. Which is surprising in light of your bug #471364. Nevertheless, ecl upstream also removed that file, so that seems to be the way to go. The merge of asdf-bundle into asdf happened for asdf 2.26.7. The last stand-alone version was identical to the one bundled with ecl, whereas the merge contains some modifications. ecl build without asdf-bundle but with the first asdf-version to include that merge already exhibits the problem for maxima. So one way would be going over that merge line by line, in an attempt to make the unmerged and the merged version even more alike, thus hopefully identifying the single change which broke things. Here is that diff:

--- asdf-bundle.lisp from asdf-bundle 37a894b12d1a3bff4131f3a4b4f7045b8ed4fbfd^
+++ fragment starting at "BUNDLE-OP" of asdf.lisp from asdf 5f7af391ecf6a2e1a43aaa442ffe74bacf66e514
@@ -1,15 +1,3 @@
-;;; -*- Mode: Lisp ; Syntax: ANSI-Common-Lisp -*-
-
-#+xcvb (module ())
-
-(in-package :asdf)
-
-#+mkcl
-(eval-when (:compile-toplevel :load-toplevel :execute)
-  ;; Make sure we have strict ANSI class redefinition semantics.
-  (setq clos::*redefine-class-in-place* t))
-
-;;;
 ;;; BUNDLE-OP
 ;;;
 ;;; This operation takes all components from one or more systems and
@@ -18,10 +6,6 @@
 ;;; The different targets are defined by specialization.
 ;;;

-(defun fasl-type ()
-  "pathname TYPE for lisp FASt Loading files"
-  (#-ecl load-time-value #+ecl identity
-   (pathname-type (compile-file-pathname "foo.lisp"))))

 (defclass bundle-op (operation)
   ((build-args :initarg :args :initform nil :accessor bundle-op-build-args)
@@ -253,6 +237,8 @@

 (defclass load-fasl-op (operation) ())

+(defgeneric trivial-system-p (component))
+
 (defmethod component-depends-on ((o load-fasl-op) (c system))
   (declare (ignorable o))
   (unless (trivial-system-p c)
@@ -287,8 +273,8 @@
 (defclass compiled-file (component)
   ((type :initform nil)))

-(defun trivial-system-p (c)
-  (every #'(lambda (c) (typep c 'compiled-file)) (module-components c)))
+(defmethod trivial-system-p ((s module))
+  (every #'(lambda (c) (typep c 'compiled-file)) (module-components s)))

 (defmethod component-relative-pathname ((component compiled-file))
   (coerce-pathname
@@ -423,10 +409,6 @@
                                :direction :input :if-does-not-exist :error)
         (copy-stream-to-stream i o :element-type '(unsigned-byte 8))))))

-(defun* add-pathname-suffix (pathname suffix)
-  (make-pathname :name (strcat (pathname-name pathname) suffix)
-                 :defaults pathname))
-
 (defun combine-fasls (inputs output)
   #-(or allegro clisp clozure cmu lispworks sbcl scl)
   (declare (ignore inputs output))
@@ -454,24 +436,6 @@
       (loop :for f :in fasls :do (ignore-errors (delete-file f)))
       (ignore-errors (lispworks:delete-system :fasls-to-concatenate)))))

-(defun call-with-staging-pathname (pathname fun)
-  "Calls fun with a staging pathname, and atomically
-renames the staging pathname to the pathname in the end.
-Note: this protects only against failure of the program,
-not against concurrent attempts.
-For the latter case, we ought pick random suffix and atomically open it."
-  (let* ((pathname (pathname pathname))
-         (staging (add-pathname-suffix pathname "-ASDF-TMP")))
-    (unwind-protect
-         (multiple-value-prog1
-             (funcall fun staging)
-           (rename-file staging pathname #+clozure :if-exists #+clozure :rename-and-delete))
-      (when (probe-file* staging)
-        (delete-file staging)))))
-
-(defmacro with-staging-pathname ((pathname-var &optional (pathname-value pathname-var)) &body body)
-  `(call-with-staging-pathname ,pathname-value #'(lambda (,pathname-var) ,@body)))
-
 #-(or ecl mkcl)
 (defmethod perform ((o bundle-op) (c system))
   (let* ((input-files (input-files o c))
@@ -625,12 +589,37 @@
 ;; the ability to build a fasb bundle, so this is somewhat unrelated to what is above.
 );mkcl

-;;;
-;;; Final integration steps
-;;;

-(export '(load-fasl-op precompiled-system
-          #+ecl make-build #+mkcl bundle-system))
+
+;;;; -----------------------------------------------------------------
+;;;; Done!
+(when *load-verbose*
+  (asdf-message ";; ASDF, version ~a~%" (asdf-version)))
+
+#+mkcl
+(handler-case
+    (progn
+      (load-sysdef "asdf-bundle"
+                   (subpathname (translate-logical-pathname #P"CONTRIB:")
+                                "asdf-bundle/asdf-bundle.asd"))
+      (load-system "asdf-bundle"))
+  (error (e)
+    (format *error-output*
+            "~&;;; ASDF: Failed to load package 'asdf-bundle'!~%;;; ~A~%" e)))
+
+#+allegro
+(eval-when (:compile-toplevel :execute)
+  (when (boundp 'excl:*warn-on-nested-reader-conditionals*)
+    (setf excl:*warn-on-nested-reader-conditionals* *acl-warn-save*)))

 #+(or ecl mkcl)
 (pushnew '("fasb" . si::load-binary) si:*load-hooks* :test 'equal :key 'car)
+
+(pushnew :asdf *features*)
+(pushnew :asdf2 *features*)
+
+(provide :asdf)
+
+;;; Local Variables:
+;;; mode: lisp
+;;; End:

Of course it might well be that the source code changes have nothing to do with the problem, but that it instead lies in some metadata, i.e. the cause is not what code is compiled but instead from where it got compiled. Not sure how such a thing could be, but given the modular structure of things, I could imagine some symbol ending up in a wrong namespace or some such.

[gagern]

Not sure why, but it seems that including these source files into ecls does something different from loading them from source using (load "/path/to/asdf.lisp"). The latter seems to work in more cases, including the one where asdf-bundle got merged. I could bisect the breakage to 2.26.71. Before this, I had to disable the maxima server module and make sure that asdf/lisp-action uses asdf/implementation, and even then I got cross-device symlinks due to lisp cache directory settings. But before this commit, I got some maxima.system.fasb built, whereas afterwards I got errors about missing component maxima at first and :MAXIMA later on.

[gagern]

I guess I got one crucial part of the puzzle: the changes to the source registry configuration, namely the default-directory removal, seems to omit the current working directory from the search. A fix for this would be -eval '(push "./" asdf:*central-registry*)'. According to the docs this is “the old way” to do things, but I couldn't get initialize-source-registry working yet, and this seems to get the job done.

With this in place, I get the cross device link issue again. Working steps against this problem appear to be dropping all this lisp-cache stuff, adding -eval "(asdf:initialize-output-translations '(:output-translations :disable-cache :inherit-configuration))" and dropping the :move-here "." from the make-build invocation.

[gagern]

ecls-12.12.1-r5 still won't compile the maxima server, though:

;;; End of Pass 1.ASDF could not load sb-bsd-sockets because
Error while trying to load definition for system sb-bsd-sockets from pathname
SYS:SB-BSD-SOCKETS.ASD.NEWEST: There is no package with the name ASDF-1.0..

Condition of type: MISSING-SYSTEM
MK:DEFSYSTEM: missing system MAXIMA::SB-BSD-SOCKETS.

I'm emerging ecls-13.5.1 now, to see whether that works any better.

[kiwifb]

I had guessed the part about the current working directory being removed from the search path but have been incapable of finding the right command to put it back. note that asdf:make-build is just kept around for backward compatibility. The new way is (asdf:operate 'load-fasl-op :mysystem) not sure how to make it work either.

[gagern]

ecls-13.5.1 does generate a maxima.fasb again, not maxima.system.fasb. Otherwise it appears to work well. I'm giving the ebuild a final run, then I'll commit it to a branch which I'll publish to my fork here on github. I'll create a pull request.

[kiwifb]

OK merged. I will alert Andrey that we have a new way to build maxima.fas and i will relax the dependencies on ecl/maxima a bit. I may have to import 12.12.1-r5 and add the few patches that are not in the main tree. I will do that over the next 48hours.

[kiwifb]

Trying to adapt the solution for maxima 5.29.1 I got End of Pass 1.ASDF could not load sb-bsd-sockets because Error while trying to load definition for system sb-bsd-sockets from pathname SYS:SB-BSD-SOCKETS.ASD.NEWEST: There is no package with the name ASDF-1.0.. Condition of type: MISSING-SYSTEM MK:DEFSYSTEM: missing system MAXIMA::SB-BSD-SOCKETS. I am guessig that will be just more patching now that I have been able to read the message in full. asdf 1.0, that's weird.

Folks try 5.30 because 5.29.1 is broken until tomorrow at least.

[gagern]

Strange, I considered this sb-bsd-sockets thing a problem from the ecls/asdf installation, unrelated to the version of maxima itself. But apart from this, is there any reason why we should prefer 5.29 over 5.30?

[kiwifb]

Not sure yet. I haven't tested it. Sometimes stuff break or need to be changed in sage. Testing was interupted by snooze time.

[kiwifb]

Plenty of doctests failures with 5.30 including maxima_lib.py. Do you have more luck than me?

[gagern]

sage -t /usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py segfaults. Adding a -gdb tells me that cygdb can't find debug info. Error message is No debug files were found in /usr/share/sage/devel/sage. Aborting. like quoted in #217. I do have FEATURES="splitdebug" on my system, but unfortunately this does not seem enough. Sad, since all I'm after is some kind of backtrace, I don't really care about debug symbols just now. Do I have to recompile sage-clib with unstripped and unsplit debug symbols, or do you see an alternative?

[kiwifb]

I have debug symbols so I'll look when i am able too. You could try sage -t /usr/lib64/python2.7/site-package/sage/interface/maxima_lib.py also.

[gagern]

Corrected that path for plurals (site-packages and interfaces) and got ImportError: No module named maxima_abstract for my troubles. Which sounds like something with the python path didn't work the way it should, since maxima_abstract.py is present in that very same directory.

[kiwifb]

I think upstream may have done something about the location of debug info in a recent patch, I am checking this.

[kiwifb]

OK, it turns out that stuff we don't currently generate - I think. I'll dig that up.

[kiwifb]

Proper handling of that stuff is in 5.10_rc1 (unrelesead). Added to the 5.10_beta5 ebuild witht he debug useflag. Now I can run the test with --gdb and get into gdb. 0x00007fffc0726f3b in do_copy_tree (x=0x6784161) at /scratch/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d:477

hundreds of thousands of times...... with slightly different address of course. Getting the debug flag in ecls

[kiwifb]

Not much improvements:

sage -t /usr/lib64/python2.7/site-packages/sage/interfaces/maxima_lib.py
Program received signal SIGSEGV, Segmentation fault.
do_copy_tree (x=0x1) at /scratch/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d:475
475     /scratch/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d: No such file or directory.
(gdb) bt full
#0  do_copy_tree (x=0x1) at /scratch/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d:475
No locals.
#1  0x00007fffc0712f2f in do_copy_tree (x=0x6784161) at /scratch/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d:477
No locals.

[kiwifb]

I have a sanitized ecls 12.12.1 that has the correct stuff for SB-BSD-SOCKETS (and all other .asd files actually). I can build maxima.fas with it. Testing sage now.

[kiwifb]

Not working. I think our new maxima.fas may be buggy or is not called the right way.

sage: from sage.libs.ecl import *
sage: ecl_eval("(setf *load-verbose* NIL)")

sage: ecl_eval("(require 'maxima)")
Segmentation fault

[gagern]

Got my sage-5.10_beta5 emerged. /usr/lib64/python2.7/site-packages/sage/interfaces/maxima_lib.py still complains No module named maxima_abstract, /usr/share/sage/devel/sage/sage/interfaces/maxima_lib.py still segfaults and with -gdb I still get No debug files were found in /usr/share/sage/devel/sage. Aborting.. sage -gdb by itself won't work either. Running sage interactively, I could attach gdb to that process, issue the commands you quoted, intercept the SIGSEGV and see those do_copy_tree calls. There were in excess of 60.000 nested calls, but they mostly involved only 9 different objects, which indicates infinite recursion there. Seems like it was copying some cyclic data structure, and not a tree at all. It apparently started around here:

#261518 0x00007f71fdbe15ef in do_copy_tree (x=0x79ef391)
#261519 0x00007f71fdbe15fb in do_copy_tree (x=0x79ef251)
#261520 0x00007f71fdbe15ef in do_copy_tree (x=0x79ef261)
#261521 0x00007f71fdbe15ef in do_copy_tree (x=0x79ef271)
#261522 0x00007f71fdbe15fb in do_copy_tree (x=0x79ef241)
#261523 0x00007f71fdbe15fb in do_copy_tree (x=0x79ef201)
#261524 0x00007f71fdbe15ef in do_copy_tree (x=0x8fc6ab1)
#261525 0x00007f71fdbe15ef in do_copy_tree (x=0x8fc69d1)
#261526 0x00007f71fdbe15ef in do_copy_tree (x=0x9256b91)
#261527 0x00007f71fdbe15fb in do_copy_tree (x=0x8287cf1)
#261528 0x00007f71fdbe15ef in do_copy_tree (x=0x8287ce1)
#261529 0x00007f71fdbe15ef in do_copy_tree (x=0x7c49cc1)
#261530 0x00007f71fdbe15ef in do_copy_tree (x=0x7c49cb1)
#261531 0x00007f71fdbe15ef in do_copy_tree (x=0x7c90861)
#261532 0x00007f71fdbe15ef in do_copy_tree (x=0x7c90851)
#261533 0x00007f71fdbe2847 in cl_copy_tree (x=<optimized out>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/list.d:485
#261534 0x00007f71fd6ac9d1 in _ecltZjroluad2YbA_A6xHB111 (flag=<optimized out>)
    at binary-ecl/init-cl.c:1739
#261535 0x00007f71fdc02879 in ecl_init_module (block=0x8f0f900, 
    entry_point=0x7f71fd6abe20 <_ecltZjroluad2YbA_A6xHB111>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/read.d:2444
#261536 0x00007f71fd6af033 in init_lib_MAXIMA (cblock=<optimized out>)
    at /var/tmp/portage/sci-mathematics/maxima-5.30.0-r1/temp/eclinityWjiO2.c:1805
#261537 init_lib_MAXIMA (cblock=0x1, cblock@entry=0x0)
    at /var/tmp/portage/sci-mathematics/maxima-5.30.0-r1/temp/eclinityWjiO2.c:311
#261538 0x00007f71fdc02879 in ecl_init_module (block=0x688b580, 
    entry_point=0x7f71fd6acec0 <init_lib_MAXIMA>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/read.d:2444
#261539 0x00007f71fd018310 in init_fas_CODE (cblock=0x1, cblock@entry=0x0)
    at /var/tmp/portage/sci-mathematics/maxima-5.30.0-r1/temp/eclinit4gz5P5.c:1811
#261540 0x00007f71fdc02879 in ecl_init_module (block=block@entry=0x6888100, 
    entry_point=0x7f71fd016180 <init_fas_CODE>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/read.d:2444
#261541 0x00007f71fdc3e475 in si_load_binary (filename=0x7c97800, 
    verbose=<optimized out>, print=<optimized out>, external_format=<optimized out>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/load.d:72
#261542 0x00007f71fdbe4b68 in cl_funcall (narg=<optimized out>, narg@entry=5, 
    function=function@entry=0x7f71fdf837e0 <cl_symbols+88928>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/eval.d:132
#261543 0x00007f71fdc3f015 in cl_load (narg=narg@entry=3, source=<optimized out>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/load.d:274
#261544 0x00007f71fdb7ffe1 in LC3__g19 (v1module=<optimized out>) at lsp/module.c:199
#261545 0x00007f71fdb800b9 in LC2__g17 (narg=<optimized out>, v1p=0x68502c0)
    at lsp/module.c:178
#261546 0x00007f71fdbe4d98 in cl_apply (narg=<optimized out>, narg@entry=2, 
    fun=fun@entry=0x7c8c780, lastarg=<optimized out>, lastarg@entry=0x7c99251)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/eval.d:166
#261547 0x00007f71fdb2a9d1 in cl_some (narg=narg@entry=2, 
    v1predicate=v1predicate@entry=0x7c8c780, v2sequence=<optimized out>) at lsp/seq.c:881
#261548 0x00007f71fdb804c2 in cl_require (narg=<optimized out>, 
    v1module_name=<optimized out>) at lsp/module.c:153
#261549 0x00007f71fdbe73a5 in ecl_interpret (frame=frame@entry=0x7fff5902ced0, 
    env=<optimized out>, bytecodes=bytecodes@entry=0x7c103c0)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/interpreter.d:501
#261550 0x00007f71fdbec291 in eval_nontrivial_form (form=0x7c103c0, env=0x7f7230817000)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/compiler.d:2374
#261551 eval_form (env=env@entry=0x7f7230817000, form=form@entry=0x7c993d1)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/compiler.d:2388
#261552 0x00007f71fdbedfa3 in si_eval_with_env (narg=<optimized out>, form=0x7c993d1)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/compiler.d:3129
#261553 0x00007f71fdbe63e9 in ecl_interpret (frame=0x7fff5902d200, env=<optimized out>, 
    bytecodes=0x7c10500)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/interpreter.d:424
#261554 0x00007f71fdbe4b68 in cl_funcall (narg=<optimized out>, narg@entry=2, 
    function=<optimized out>)
    at /var/tmp/portage/dev-lisp/ecls-13.5.1/work/ecl-13.5.1/src/c/eval.d:132
#261555 0x00007f71fdfa6c6d in __pyx_f_4sage_4libs_3ecl_ecl_safe_eval (
    __pyx_v_form=0x7c993d1) at build/cythonized/sage/libs/ecl.c:3053
#261556 0x00007f71fdfa71ff in __pyx_f_4sage_4libs_3ecl_ecl_eval (__pyx_v_s=
    "(require 'maxima)", __pyx_skip_dispatch=<optimized out>)
    at build/cythonized/sage/libs/ecl.c:7257

This address of 0x00007f71fd6ac9d1 comes from maxima.fas. Will have to have a closer look at the build process and try to get hold of both this init-cl.c (apparently the result of init-cl.lisp) and this temporary file eclinityWjiO2.c, so I can map these line numbers back to lisp code. But not today.

[kiwifb]

There has been change to the command line options, use --gdb. double dash gnu style.

[kiwifb]

Where did you get the complaint about maxima_abstract?

[strogdon]

Perhaps the problem is deeper than maxima.fas. For a little more verbose output

sage: from sage.libs.ecl import *
sage: ecl_eval("(require 'maxima)")
;;; Loading #P"/usr/lib64/ecl-13.5.1/maxima.fas"
;;; Loading #P"/usr/lib64/ecl-13.5.1/sb-bsd-sockets.fas"
;;; Loading #P"/usr/lib64/ecl-13.5.1/sockets.fas"
Segmentation fault

[kiwifb]

Yes, I got that output from ecl directly.

ecl
ECL (Embeddable Common-Lisp) 12.12.1 (git:UNKNOWN)
Copyright (C) 1984 Taiichi Yuasa and Masami Hagiya
Copyright (C) 1993 Giuseppe Attardi
Copyright (C) 2000 Juan J. Garcia-Ripoll
ECL is free software, and you are welcome to redistribute it
under certain conditions; see file 'Copyright' for details.
Type :h for Help.  
Top level.
> (require 'maxima)
;;; Loading #P"/usr/lib64/ecl-12.12.1/maxima.fas"
;;; Loading #P"/usr/lib64/ecl-12.12.1/sb-bsd-sockets.fas"
;;; Loading #P"/usr/lib64/ecl-12.12.1/sockets.fas"
Segmentation fault

maxima is 5.30, but 5.29-r2 does the same. You cannot do that with ecls 12.12.1-* in tree or overlay, it needs an extra cherry picked patch from upstream. But the fact that I get the same thing with ecl 12.12.1 means that something in the new asdf-bundle is the trigger. It may be a bug that was previously hidden but asdf-3.x is why we see it now.

[gagern]

The maxima_abstract message was for sage -t /usr/lib64/python2.7/site-packages/sage/interfaces/maxima_lib.py like I already wrote eralier in reply to your suggestion of that path.

With regard to this more verbose output: I don't think sockets.fas is involved: the backtrace doesn't support such a hypothesis. I believe that sockets.fas was the last file for which loading started, but it also finished successfully, whereas subsequent initialization of maxima.fas fails.

[kiwifb]

I don't remember ever having a message for maxima_abstract. That's odd. I'll check it later. Steve Do you have a working sage-5.9 on which we could see the full verbose output?

[gagern]

By the way, is there any reasony why the creation of maxima.fasb should be in src_install and not src_compile? Seems odd to me.

The function _ecltZjroluad2YbA_A6xHB111 in my above stack trace does not appear to directly correspond to any lisp code from init-cl.lisp. Instead it is part of some code emitted after all the functions from that file, presumably initialization code for that translation unit. Perhaps dynamic linking. Here is a bit of context for the code in question:

L275:;
    value0 = ecl_function_dispatch(cl_env_copy,v2)(0);
    v3 = value0;
    {
     const int v5 = cl_env_copy->nvalues;
     cl_object v6;
     v6 = (v5<=1)? ECL_NIL : cl_env_copy->values[1];
     v4s = v6;
    }
    if ((v3)!=ECL_NIL) { goto L280; }
    goto L269;
L280:;
    T0 = cl_symbol_package(v4s);
    if (!((T0)==(v1maxima_package))) { goto L282; }
    if ((v4s)==(VV[232])) { goto L282; }
    T0 = ecl_symbol_name(v4s);
    {
     ecl_character v5;
     v5 = ecl_char(T0,0);
     T1 = (ECL_SYM("CHAR=",219)->symbol.gfdef);
     if (Null(cl_member(4, CODE_CHAR(v5), VVtemp[16], ECL_SYM("TEST",1316), T1))) { goto L282; }
    }
    cl_set(VV[233],CONS(v4s,ecl_symbol_value(VV[233])));
    T0 = cl_symbol_plist(v4s);
    T1 = cl_copy_tree(T0);
    si_hash_set(v4s, ecl_symbol_value(VV[234]), T1);
L282:;
L276:;
    goto L275;
   }
  }
L269:;

Not sure what data structure that might correspond to. I got this file by invoking make with an LD_PRELOAD interposter library which moves files instead of unlinking them. The symbol name in question has some random salt in it, but you can get your local name using

objdump -T /usr/lib/ecl-*/maxima.fas | awk '/ecltZjroluad2YbA/{print $7}'

You can also simply add a breakpoint at binary-ecl/init-cl.c:1739 to get to this position. Stepping into the cl_copy_tree call, one can get a closer look at this argument x. But deparsing that manually is tedious at best. I guess I'll try to come up with some PythonGdb script to print ecl data structures in a nice way, avoiding infinite recursion in this special case here. That should give us an idea as to what object we are actually handling here, and where the loop is.

[gagern]

Using this code, I managed to print the data structure in question:

( MAXIMA::SYSCONST                                      ; 0x2899b80
  COMMON-LISP::T                                        ; 0x3e61afacb8 <cl_symbols+56>
  MAXIMA::TEXWORD                                       ; 0x355c480
  '\\gamma'                                             ; 0x3634ae0
  MAXIMA::DATA                                          ; 0x27c3b00
  ( ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$CONSTANT                               ; 0x2860d80
      )                                                 ; 0x326ae31
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0x326ae01
    ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$REAL                                   ; 0x2860900
      )                                                 ; 0xd76901
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0xd768c1
    ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$NONINTEGER                             ; 0x28a6880
      )                                                 ; 0xd76a41
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0xd76a11
    ( ( MAXIMA::MEQP                                    ; 0x205ffc0
    ^   MAXIMA::$%GAMMA                                 ; 0x28a2000
    |   ( 0.57721566490153287                           ; 0x35d52b0
    |   ^ MAXIMA::DATA                                  ; 0x27c3b00
    \---|-( <recursive!>                                ; 0x2133f61
        |   ( ( MAXIMA::MGRP                            ; 0x2b185c0
        |   ^   ( 1.6180339887498949                    ; 0x35d52e0
        |   |   ^ MAXIMA::DATA                          ; 0x27c3b00
        |   \---|-( <recursive!>                        ; 0x2132041
        |       |   ( ( MAXIMA::MEQP                    ; 0x205ffc0
        |       |       MAXIMA::$%PHI                   ; 0x2860440
        |       +------ <recursive!>                    ; 0x21324c1
        |       |     )                                 ; 0x21323d1
        |       |     MAXIMA::CON                       ; 0x31404c0
        |       |     MAXIMA::$INITIAL                  ; 0x2dcd640
        |       |   )                                   ; 0x21323a1
        |       |   ( ( MAXIMA::MGRP                    ; 0x2b185c0
        |       |   ^   ( 2.7182818284590451            ; 0x35d5340
        |       |   |   ^ MAXIMA::DATA                  ; 0x27c3b00
        |       |   \---|-( <recursive!>                ; 0x2132481
        |       |       |   ( ( MAXIMA::MGRP            ; 0x2b185c0
        |       |       |   ^   ( 3.1415926535897931    ; 0x35d5310
        |       |       |   |   ^ MAXIMA::DATA          ; 0x27c3b00
        |       |       |   |   | ( ( ( MAXIMA::MEQP    ; 0x205ffc0
        |       |       |   |   |       MAXIMA::$%PI    ; 0x27eaa00
        |       |       |   |   \------ <recursive!>    ; 0x2132941
        |       |       |   |         )                 ; 0x2132831
        |       |       |   |         MAXIMA::CON       ; 0x31404c0
        |       |       |   |         MAXIMA::$INITIAL  ; 0x2dcd640
        |       |       |   |       )                   ; 0x2132801
        |       |       |   \------ <recursive!>        ; 0x2132901
        |       |       |         )                     ; 0x21327b1
        |       |       |       )                       ; 0x2132941
        |       |       +------ <recursive!>            ; 0x2132d11
        |       |       |     )                         ; 0x2132931
        |       |       |   )                           ; 0x2132901
        |       |       |   ( ( MAXIMA::MEQP            ; 0x205ffc0
        |       |       |       MAXIMA::$%E             ; 0x27eaa40
        |       |       \------ <recursive!>            ; 0x2132d11
        |       |             )                         ; 0x2132cf1
        |       |             MAXIMA::CON               ; 0x31404c0
        |       |             MAXIMA::$INITIAL          ; 0x2dcd640
        |       |           )                           ; 0x2132cc1
        |       |         )                             ; 0x2132471
        |       |       )                               ; 0x2132d11
        |       \------ <recursive!>                    ; 0x21324c1
        |             )                                 ; 0x21324b1
        |           )                                   ; 0x2132481
        |         )                                     ; 0x2132031
        |       )                                       ; 0x21324c1
        \------ <recursive!>                            ; 0x2132081
              )                                         ; 0x2132071
            )                                           ; 0x2132041
          )                                             ; 0x2133f11
        )                                               ; 0x2132081
      )                                                 ; 0x2133f91
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$INITIAL                                  ; 0x2dcd640
    )                                                   ; 0x2133f61
    ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$CONSTANT                               ; 0x2860d80
      )                                                 ; 0x27e9021
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0x27ecff1
    ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$REAL                                   ; 0x2860900
      )                                                 ; 0x2d1dc61
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0x2d1dc31
    ( ( MAXIMA::KIND                                    ; 0x289eb00
        MAXIMA::$%GAMMA                                 ; 0x28a2000
        MAXIMA::$NONINTEGER                             ; 0x28a6880
      )                                                 ; 0x2d1dd61
      MAXIMA::CON                                       ; 0x31404c0
      MAXIMA::$GLOBAL                                   ; 0x2dcd7c0
    )                                                   ; 0x2d1dd31
  )                                                     ; 0x326adc1
  MAXIMA::IMPLIED-QUOTEP                                ; 0x200f140
  COMMON-LISP::T                                        ; 0x3e61afacb8 <cl_symbols+56>
  MAXIMA::MPROPS                                        ; 0x27684c0
  ( nil
    MAXIMA::$NUMER                                      ; 0x271a940
    0.57721566490153287                                 ; 0x36825b0
  )                                                     ; 0x291d151
)                                                       ; 0x27ecf91

The address printed at the closing parenthesis of a list is the address of the first cons at the head of the list, but I've added some arrows to the corresponding opening paren to make things clearer. Can you tell what part of maxima this might be coming from? What is it that's being initialized here? Who built this recursive data structure, and is it intentionally recursive or by accident?

[gagern]

The following part of init-cl.lisp might be involved:

(eval-when (:load-toplevel :execute)
    (let ((context '$global))
      (declare (special context))
      (dolist (x '($%pi $%i $%e $%phi %i $%gamma  ;numeric constants
                   $inf $minf $und $ind $infinity ;pseudo-constants
                   t nil))                        ;logical constants (Maxima names: true, false)
        (kind x '$constant)
        (setf (get x 'sysconst) t))))

It does mention several of the symbols that occur in my data dump, and it does seem to execute at module load time. On the other hand, it looks pretty sane, so I don't know why that would garble data structures the way it does. And I fear that the code might be useful, so simply disabling it will probably have adverse effects.

[gagern]

I should have read on a few more lines in init-cl.lisp:

;;; The assume database structures for numeric constants such as $%pi and $%e
;;; are circular.  Attempting to copy a circular structure
;;; into *builtin-symbol-props* would cause a hang.  Therefore
;;; the properties are copied into *builtin-symbol-props* before
;;; initializing the assume database.
(let ((maxima-package (find-package :maxima)))
  (do-symbols (s maxima-package)
    (when (and (eql (symbol-package s) maxima-package)
           (not (eq s '||))
           (member (char (symbol-name s) 0) '(#\$ #\%) :test #'char=))
      (push s *builtin-symbols*)
      (setf (gethash s *builtin-symbol-props*)
        (copy-tree (symbol-plist s))))))

The test and char= symbols are included in the relevant portion of init-cl.c, right before the problematic copy-tree invocation. And the comment states that cyclic data structures will be a problem in this context, but that at that point the data structure should not be cyclic yet. So there are several possibilities: either there is a bug in maxima, causing that data structure to be cyclic at that point already. Or there is a bug in ecl, causing the creation of a cyclic data structure incorrectly. Or there might be something wrong with aspf, causing things to get initialized twice, so that during the second initialization this copy is performed on data already initialized from the first run. Personally I consider this last case the most likely one. Not sure how to proceed.

[kiwifb]

The bit about maxima.fas being built in src_install should probably be fixed. I cannot remember if there was a problem putting it in src_compile.

The odd thing I have noticed is with the new procedure maxima seem to be rebuilt to produce maxima.fas, the old way it was just repacked I think. May be we have double initialisation because we have double maxima?

[gagern]

I noticed that maxima is rebuilt because my main maxima setup is for sbcl. There is no REQUIRED_USE to restrict the configured choice of lisp implementations to exactly one. Which means that the deps and main build will pick the first enabled one from their list, but the ecl stuff will get built in any case. Without so much as a dep, as far as I can tell. I know REQUIRED_USE is sometimes frowned upon, but I'd consider this a prime case where it would be useful, both to prevent surprises to the user and to make writing depending code like sage easier.

[kiwifb]

All selected lisp configuration of maxima will be built. You can have maxima compiled for several lisp and it is switched on when you start maxima. By default it goes with sbcl but maxima -l ecl will start with ecls and so on. You can also select it with an environement variable. They all cohabit peacefully both at compile and run time so there is no reason to restrict lisp apart from making maxima compile faster.

[strogdon]

This is what I had always thought to be the case and on my work computer I have always had sbcl installed. Perhaps not important, but here with maxima-5.30.0 it is defaulting to clisp?

maxima
Maxima 5.30.0 http://maxima.sourceforge.net
using Lisp CLISP 2.49 (2010-07-07)
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1) 

maxima -l sbcl
Maxima 5.30.0 http://maxima.sourceforge.net
using Lisp SBCL 1.1.6
Distributed under the GNU Public License. See the file COPYING.
Dedicated to the memory of William Schelter.
The function bug_report() provides bug reporting information.
(%i1)

[kiwifb]

Actually I am not certain about the picking order by default. I thought it was sbcl but it could be clisp. Of course they can change it whenever they like.