search

void-linux / void-packages

The Void source packages collection
https://voidlinux.org
Other
2.59k stars 2.16k forks source link

maxima fails to (natively) build using ecl #46293

Closed dkwo closed 1 year ago

dkwo commented 1 year ago

Is this a new report?

Yes

System Info

6.5 series kernel on apple silicon (aarch64), as well as Void 6.5.5_2 x86_64 GenuineIntel uptodate rrFFFFFFFFFFFFFF

Package(s) Affected

maxima-5.47.0

Does a report exist for this bug with the project's home (upstream) and/or another distro?

https://sourceforge.net/p/maxima/bugs/4188/

but this is possibly due to our own https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/use-shared-library.patch

Expected behaviour

The prior version 5.46.0 builds fine.

Actual behaviour

In the do_build phase:

Making all in info
make[2] Entering directory '/builddir/maxima-5.47.0/doc/info'
make[3] Entering directory '/builddir/maxima-5.47.0/doc/info'
/usr/bin/env perl build_index.pl maxima.info ':crlf' > maxima-index.lisp
../../maxima-local --batch-lisp=build-html-index.lisp
/builddir/maxima-5.47.0/src/binary-ecl/maxima: error while loading shared libraries: binary-ecl/libmaxima-ecl.so: cannot open shared object file: No such file or directory
make[3]: *** [Makefile:1653: maxima-index-html.lisp] Error 127

Steps to reproduce

build natively on any of those archs using ./xbps-src pkg -o ecl,~sbcl maxima

dkwo commented 1 year ago

Downgrading to 5.46.0 fixes the issue.

dkwo commented 1 year ago

@tornaria is downgrading an option? will it interfere with sage?

dkwo commented 1 year ago

./xbps-src pkg -o ecl maxima builds fine on x86_64. on a side note, we carry matrixexp.patch (which sagemath does not, but they carry infodir.patch which we don't).

tornaria commented 1 year ago

./xbps-src pkg -o ecl maxima builds fine on x86_64.

Not sure I understand what you mean. The default for x86_64 is ecl + sbcl. For aarch64 the default is only ecl. The sbcl binary is faster, so when building ecl + sbcl the installer will set sbcl as the default. I.e., when you run just maxima you are running sbcl; if you run maxima -l ecl you are running ecl. The ecl binary we move to the maxima-ecl package, so if building only ecl maybe maxima won't work without the maxima-ecl package installed (can you check that? if so we can try to do something about it).

OTOH, when you say "downgrading to 5.46.0 fixes the issue", does that mean that you rebuilt using 5.46.0, or that you used an old xbps file? If the latter, maybe the issue is not with maxima itself, but with some change in ecl which was recently updated. Maybe there is something wrong with ecl on aarch64? Can you try different combinations:

on a side note, we carry matrixexp.patch (which sagemath does not, but they carry infodir.patch which we don't).

I don't think the infodir.patch does anything outside windows. I believe we carry matrixexp.patch because its still a actual bug worth fixing. IIRC the sagemath crowd decided to workaround the bug in sage itself and thus dropped the patch (since most maxima available in distros have the bug).

dkwo commented 1 year ago

Let me clarify: the error was in do_build phase of maxima itself, not maxima-ecl. Even on x86_64, ./xbps-src pkg -o ecl,~sbcl maxima fails in the same way, so this is not aarch64-related, rather ecl-related. After rebuilding ecl natively, nothing changes. maxima is currently nocross (not sure why), so I was rebuilding it with 5.46.0 and confirmed it builds fine. Thanks for the clarification about patches.

does this make sense?

robert-dodier commented 1 year ago

Hi, I'm a Maxima developer and I'm working on the upstream bug report. @tornaria you mentioned matrixexp.patch; where can I find that patch? I will take a look at merging it into Maxima.

dkwo commented 1 year ago

Thanks for helping. The patches are here: https://github.com/void-linux/void-packages/tree/master/srcpkgs/maxima/patches

gunterkoenigsmann commented 1 year ago

Might the problem be that we use the (potentially cross-built) Maxima during the build process, possibly not telling it where to find it's libraries?

robert-dodier commented 1 year ago

@dkwo Thanks for linking the patches, I will take a look at them.

About libmaxima-ecl.so, I seem to recall now that it's part of the Maxima installation for Sage. Is the Maxima build machinery that you are working with somehow derived from the stuff for Sage? If so, I guess my advice is to start over with a vanilla installation. Is interoperability with Sage a requirement for a Maxima-only installation? I don't know what the requirements for Void Linux are, you can tell me.

gunterkoenigsmann commented 1 year ago

Are you cross-compiling for the arm platform? AFAIK the build process for the current Maxima involves starting Maxima...

dkwo commented 1 year ago

@gunterkoenigsmann I'm building natively, both for the x86_64 and aarch64 archs.

@robert-dodier I'm building just maxima here, not as part of sage. it's downloaded from ${SOURCEFORGE_SITE}/maxima/maxima-${version}.tar.gz and built using the gnu-configure build-style in https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/template (the only modification are the patches I linked before, as well as the

post_configure() {
    # do not rebuild these files if they exist
    touch -c doc/info/*.html
    touch -c doc/info/maxima.info*
    touch -c doc/info/maxima_toc.html
    touch -c interfaces/xmaxima/doc/xmaxima.html
}

which I'm not sure is relevant here.

Let me emphasize that the failure only happens with maxima 5.47 (while 5.46 builds fine) and only when building with ecl (if using sbcl, it builds fine).

dkwo commented 1 year ago

the gnu-configure build style runs standard

do_configure() {
    : ${configure_script:=./configure}

    export lt_cv_sys_lib_dlsearch_path_spec="/usr/lib64 /usr/lib32 /usr/lib /lib /usr/local/lib"
    ${configure_script} ${configure_args}
}
do_build() {
    : ${make_cmd:=make}

    export lt_cv_sys_lib_dlsearch_path_spec="/usr/lib64 /usr/lib32 /usr/lib /lib /usr/local/lib"
    ${make_cmd} ${makejobs} ${make_build_args} ${make_build_target}
}
dkwo commented 1 year ago

Removing all the patches and the post_configure phase still fails.

dkwo commented 1 year ago

let me attach the config.log maxima.config.log and the build log maxima-build.log

robert-dodier commented 1 year ago

Thanks for the info. Sorry for the trouble, but can you post the content of src/autoconf-variables.lisp also?

robert-dodier commented 1 year ago

By the way, FWIW, I just built current Maxima from Git with ECL 23.9.9 on an Ubuntu 16.04 system. ldd reports:

$ ldd src/binary-ecl/maxima
    linux-vdso.so.1 =>  (0x00007fff35be6000)
    libecl.so.23.9 => /usr/local/lib/libecl.so.23.9 (0x00007f1d77502000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f1d771f9000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f1d76e2f000)
    libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007f1d76baf000)
    libgc.so.1 => /usr/lib/x86_64-linux-gnu/libgc.so.1 (0x00007f1d7693d000)
    libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f1d76735000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f1d76518000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f1d76314000)
    /lib64/ld-linux-x86-64.so.2 (0x00007f1d77aea000)

I am more than a little puzzled by the presence of libmaxima-ecl.so in the output you cited ... I don't understand how the reference to that library is being generated.

dkwo commented 1 year ago

No worries. Here 's my src/autoconf-variables.lisp:

$ cat masterdir/builddir/maxima-5.47.0/src/autoconf-variables.lisp 
; -*- Lisp -*-
(in-package :maxima)

(defparameter *autoconf-prefix* "/usr")
(defparameter *autoconf-exec_prefix* "/usr")
(defparameter *autoconf-package* "maxima")
(defparameter *autoconf-version* "5.47.0")
(defparameter *autoconf-libdir* "/usr/lib64")
(defparameter *autoconf-libexecdir* "/usr/libexec")
(defparameter *autoconf-datadir* "/usr/share")
(defparameter *autoconf-infodir* "/usr/share/info")
(defparameter *autoconf-host* "x86_64-unknown-linux-gnu")
;; This variable is kept for backwards compatibility reasons:
;; We seem to be in the fortunate position that we sometimes need to check for windows.
;; But at least until dec 2015 we didn't need to check for a specific windows flavour.
(defparameter *autoconf-win32* "false")
(defparameter *autoconf-windows* "false")
(defparameter *autoconf-ld-flags* "-Wl,-z,relro -Wl,-z,now -Wl,--as-needed    ")

;; This will be T if this was a lisp-only build
(defparameter *autoconf-lisp-only-build* (eq t 'nil))

(defparameter *maxima-source-root* "/builddir/maxima-5.47.0")
(defparameter *maxima-default-layout-autotools* "true")

Do you think it makes sense to also upstream this patch? https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/handle-multiple-ldflags.patch

Actually, could this patch be creating troubles? https://github.com/void-linux/void-packages/blob/master/srcpkgs/maxima/patches/use-shared-library.patch

dkwo commented 1 year ago

Yes, after removing the use-shared-library.patch the build goes through! it fails at a later stage, but that's expected:

=> maxima-5.47.0_3: running post_build ...
patchelf: getting info about 'src/binary-ecl/libmaxima-ecl.so': No such file or directory
=> ERROR: maxima-5.47.0_3: post_build: 'patchelf --remove-rpath src/binary-ecl/libmaxima-ecl.so' exited with 1
=> ERROR:   in post_build() at srcpkgs/maxima/template:55

@tornaria can you comment?

dkwo commented 1 year ago

removing that patch, as well as

--- a/srcpkgs/maxima/template
+++ b/srcpkgs/maxima/template
@@ -52,7 +52,6 @@ post_configure() {
 post_build() {
        if [ "$build_option_ecl" ]; then
                # everything will go in the same directory, use rpath=$ORIGIN
-               patchelf --remove-rpath src/binary-ecl/libmaxima-ecl.so
                patchelf --set-rpath \$ORIGIN \
                        src/binary-ecl/{maxima,maxima.fas}
                patchelf \
@@ -135,7 +134,6 @@ maxima-ecl_package() {
        pkg_install() {
                BINARY_ECL=/usr/lib/maxima/${version}/binary-ecl
                vmove ${BINARY_ECL}
-               vinstall src/binary-ecl/libmaxima-ecl.so 755 ${BINARY_ECL}
                vinstall src/binary-ecl/maxima.fas 755 ${BINARY_ECL}

                # symlink maxima.fas in ECLDIR

passes checks with -Q.

tornaria commented 1 year ago

The requirement for us is:

This is achieved by creating both a binary executable maxima and a binary module maxima.fas. They are both installed in /usr/lib/maxima/5.47.0/binary-ecl/.

This is supported by maxima just fine. However, the two files produced maxima and maxima.fas both contain the whole of maxima code.

Now let me explain the purpose of the use-shared-library.patch and the libmaxima-ecl.so dynamic library. By default, maxima will statically link everything so all the code is duplicated between maxima and maxima.fas. What my patch does is to place all the code in libmaxima-ecl.so and have maxima and maxima.fas dynamically load that one.

So now I think I have a clue of what is the trouble. It seems maxima needs to run maxima to finish building. When maxima is built for sbcl + ecl, everything is fine (maxima is run via sbcl). But when maxima is built for ecl, the dynamic library is not found. We may be able to fix that so the maxima executable actually works at build time (maybe using LD_LIBRARY_PATH ?).

dkwo commented 1 year ago

@robert-dodier Thanks for helping with this, and sorry for bothering you.

robert-dodier commented 1 year ago

@dkwo No problem, I'm glad to help. Looks like you have figured out what the basic problem is, good luck from here on out.

tornaria commented 1 year ago

@dkwo No problem, I'm glad to help. Looks like you have figured out what the basic problem is, good luck from here on out.

@robert-dodier sorry for the confusion with my bad use-shared-library.patch. I've since improved it so the binary that is built works without having to fiddle with patchelf.

Please let me know if I can help to upstream some of the patches we have in https://github.com/tornaria/void-packages/tree/maxima/srcpkgs/maxima/patches. I think at least the following would be quite reasonable to upstream:

Also a comment about documentation. The tarball includes all the documentation so it is not necessary to rebuild. However:

gunterkoenigsmann commented 1 year ago

If you were willing to file them at https://sourceforge.net/p/maxima/patches/ it would make them more visible for the Maxima developers