search

kpeeters / cadabra2

A field-theory motivated approach to computer algebra.
https://cadabra.science/
GNU General Public License v3.0
230 stars 37 forks source link

building 2.3.5 from source / trying to build a debian package #61

Closed alexmyczko closed 3 years ago

alexmyczko commented 6 years ago

Hi Kasper

Building seems to work, however tests fail. Here's the full log: https://people.phys.ethz.ch/~myczko/debian/cadabra2/

Should I just ignore the failing test, do you have a patch?

alexmyczko commented 4 years ago

would be great to get this solved for https://release.debian.org/bullseye/freeze_policy.html

kpeeters commented 4 years ago

Agreed, but I need someone to volunteer to push this. I have repeatedly try to get people on the Debian list to package this, but without luck. You don't actually have to package it yourself, just try to find someone on the Debian team willing to do it.

alexmyczko commented 4 years ago

i will give it another try :) you can even do yourself on mentors.debian.net

could you point me to where/how/when you tried?

https://mentors.debian.net/package/cadabra2/

remember? also did cadabra (1.x some time ago) https://qa.debian.org/developer.php?email=kasper.peeters%40aei.mpg.de I'll ask if i can stuff it into the Debian Science Team...

kpeeters commented 4 years ago

That person (gurkan) has not responded to emails for quite some time now. I do not have the resources to fix the errors in his package, and most likely it will need some tuning for the current Cadabra anyway. But that's indeed the right starting point.

alexmyczko commented 4 years ago

That person would be me. ??

kpeeters commented 4 years ago

Ah, having two different names certainly doesn't make things easier on my end ;-) Apologies. Does that email address at phys.ethz.ch not work anymore?

kpeeters commented 4 years ago

So what would you like me to do source-wise? Do you want a formal tagged release that you can build off? Anything related to those lintian errors which I can handle upstream?

alexmyczko commented 4 years ago

sure the email address works gurkan@phys.ethz.ch ?

Things like these would be great if you could handle them (if you think it's ok)... https://lintian.debian.org/tags/arch-dependent-file-in-usr-share.html https://lintian.debian.org/tags/custom-library-search-path.html https://lintian.debian.org/tags/shared-library-lacks-version.html https://lintian.debian.org/tags/bin-sbin-mismatch.html (because i use your install target, and not the one of debian/*install, unless i do, i don't know off head right now)

manual pages for? cadabra2-cli cadabra2latex

kpeeters commented 4 years ago

Had a quick peek at that lintian list. The arch-dependent-file-in-usr-share was fixed sometime in the 2.3.1.x series. I have just removed the RPATH settings as that was only relevant when the client/server/texengine functionality was located in a separate shared library; it's now all linked statically (since no other programs make use of it anyway and we do not export the functionality in public header files either). Pushed to github. The lacks-ldconfig-trigger is probably something for you to sort out in the package scripts.

Will walk through the rest later today. Will also drop you an email to see what goes wrong there.

kpeeters commented 4 years ago

Manual pages added.

kpeeters commented 4 years ago

I am not sure about the shared-library-lacks-version. My installation procedure does not install the Python module cadabra2.so in that location. Need to check why it does that on Debian. I am also not sure that I can version Python modules (i.e. make it cadabra2-2.3.1.so) without Python getting confused about import cadabra2.

kpeeters commented 4 years ago

I have removed the copies of jquery.js and hyphenator.js, which are both no longer necessary. That should sort out a few of the other lintian warnings.

alexmyczko commented 4 years ago

sounds good can try a current version and once it is good a tag/releasewould be helpful.

kpeeters commented 4 years ago

Yes, please give it another shot with current master on github; I think most issues are resolved now, would like to see whether the errors related to the cadabra2.so file still persist.

kpeeters commented 4 years ago

Please remove all references to kasper.peeters@aei.mpg.de; that address is no longer valid (hasn't been for many years). Please use kasper.peeters@cadabra.science instead.

alexmyczko commented 4 years ago

same url, upload #4 (will appear shortly), looks much better

kpeeters commented 4 years ago

Sorry about those manual pages, fixed that. As far as the rest is concerned:

E lacks-ldconfig-trigger
    lib/x86_64-linux-gnu/cadabra2.so

This does not make sense. I do not understand why it installs in lib/x86_64-linux-gnu. The installation is into PYTHON_SITE_PATH, which on my installation of bullseye gets resolved to /usr/local/lib/python3.8/site-packages (when I do not set an installation prefix obviously). Can you send me the output of the cmake stage?

W no-manual-page
    usr/bin/cadabra2-cli
    usr/bin/cadabra2latex

So that's fixed now.

W privacy-breach-generic
    usr/share/cadabra2/notebook.html [<link rel="stylesheet" href="https://cadabra.science/static/fonts/serif/cmun-serif.css">] (https://cadabra.science/static/fonts/serif/cmun-serif.css)

Cannot change this; the notebook.html file is used to generate standalone HTML files for notebooks, and the only way to pull in the font is to refer to an external URL. 

W shared-library-lacks-version
    lib/x86_64-linux-gnu/cadabra2.so cadabra2.so

Because cadabra2.so is a Python module, it cannot be versioned; if you call this cadabra2-2.3.1.so Python will refuse to import when you write import cadabra2. So I cannot change this.

X bin-sbin-mismatch
    usr/bin/cadabra2 bin/cadabra2 -> usr/bin/cadabra2

This presumably is because the configure script replaces the first line of the cadabra2 script with the path to the Python interpreter which the cmake stage found. I don't see what is the problem, to be honest.

I no-symbols-control-file
    lib/x86_64-linux-gnu/cadabra2.so

You don't want that for a Python module.

I package-contains-documentation-outside-usr-share-doc
    usr/share/cadabra2/latex/preamble.tex
    usr/share/cadabra2/notebook.html
    usr/share/cadabra2/notebook.tex

This is not documentation, but rather a bunch of template files used to generate standalone HTML and LaTeX versions of Cadabra notebook files. So I do not want to move them into /usr/share/doc.

I spelling-error-in-binary
    usr/bin/cadabra2-gtk libary library

Now fixed.

kpeeters commented 4 years ago 
X bin-sbin-mismatch
    usr/bin/cadabra2 bin/cadabra2 -> usr/bin/cadabra2

Ah, no, this is because I use a string replacement to get the package path from the script path, and that replace contains /bin/cadabra2 as the search pattern. So it does not actually refer to /bin/cadabra2 at all; lintian's conclusion is wrong.

alexmyczko commented 4 years ago

build log is here

http://phd-sid.ethz.ch/debian/cadabra2/2020/

kpeeters commented 4 years ago

That shows that cadabra2.so is installed in

/www/debian/cadabra2/2020/cadabra2-2.3.0+git20201030/debian/cadabra2/usr/lib/x86_64-linux-gnu/python3.9/site-packages/cadabra2.so

Not sure why lintian claims otherwise.

Can we ignore these warnings related to cadabra2.so or does the package only get accepted when all warnings have been removed?

alexmyczko commented 4 years ago

no lintian can be wrong and we can set overrides to ignore them, should be reporte for improving lintian...

kpeeters commented 4 years ago

Ok. I would suggest we then do a formal tagged release now so you can push it further. This will be 2.3.2. I'll let you know when it's ready (want to fold in a few other changes).

alexmyczko commented 4 years ago

I've updated the mentors page with a today version and the manual pages are all fine now. There's this one E: and two W: to fix left...

kpeeters commented 4 years ago

That 'E' is for you, no? 

 Otherwise, please add activate-noawait ldconfig to the triggers file in the control member.

The two 'W's need to be overridden; I cannot remove that link to cadabra's own web site (it would break standalone html versions of notebooks), and I cannot version python module shared libraries (it won't import anymore if we do that).

kpeeters commented 4 years ago

Github is running a test build for 2.3.2 on Linux & macOS now; when that's done and all tests pass I'll tag and release.

kpeeters commented 4 years ago

Ok, release 2.3.2 is there.

alexmyczko commented 4 years ago

looks much better now, see mentors link ;)

kpeeters commented 4 years ago

Very good, thanks for pushing this. Do you know when this will hit the bullseye repos so we can test the final product?

alexmyczko commented 4 years ago

Well once one of the 1000 official debian developers (one with upload rights) finds the source package, feels he needs/wants it, uploads it to debian new queue, it gets eventually processed by ftp master/team (accept or reject), then in debian sid/unstable it wanders to testing/bullseye within 2-10 days, if rc bugless (but only with a -2 source only upload) :)

kpeeters commented 4 years ago

I just noticed that when building on Debian/Ubuntu with CMAKE_INSTALL_PREFIX set to /usr, the cadabra2.so file will go into

/usr/lib/x86_64-linux-gnu/python3.6/site-packages/

(or some other version of python depending on the distro). The important part here is the arch-specific lib directory, which I think is correct because cadabra2.so is a binary, not a source-code Python module.

However, the above directory is not in Python's search path on Debian/Ubuntu. So if you do

python3 -c 'import cadabra2'

it fails.

What's your opinion on this? Should we install instead in

/usr/lib/python3.6/site-packages/

or perhaps even in

/usr/lib/python3/dist-packages

when we install from a proper Debian/Ubuntu package? Those two are both in Python's search path on Debian/Ubuntu.

kpeeters commented 3 years ago

@alexmyczko I checked this with debian-mentors, and the reply was pretty clear (https://lists.debian.org/debian-mentors/2020/11/msg00156.html): the cadabra2.so file needs to go into /usr/lib/python3.x/dist-packages/ and should carry the python SO ABI name, so e.g. cadabra2.cpython-36m-x86_64-linux-gnu.so. I have changed the various CMakeLists.txt to conform to this. You may want to check that this works for you.

alexmyczko commented 3 years ago

Here's the results: http://phd-sid.ethz.ch/debian/cadabra2/2121/ (it's still building)

kpeeters commented 3 years ago

Except that 2.3.2 does not have that yet ;-( Sorry, I forgot to tag 2.3.5, it's done now.

alexmyczko commented 3 years ago

it was a master checkout from today... not visible in the version with git20201117? - going for 2.3.5 then... - http://phd-sid.ethz.ch/debian/cadabra2/2121/ as usual (still building)

alexmyczko commented 3 years ago

@d-torrance can i ping you here? ok after many rounds of d/copyright i feel i have no more power to finish it off, main issue left is frontend/common/* for d/copyright if you want to pick it up?

kpeeters commented 3 years ago

@alexmyczko @d-torrance If this needs any help from my end, please let me know, would be a bit of a pity to let all the work done so far go to waste.

alexmyczko commented 3 years ago

https://mentors.debian.net/package/cadabra2/ i must fix the debian/copyright file

jfyi https://ftp-master.debian.org/new/cadabra2_2.3.5-1.html

kpeeters commented 3 years ago

@alexmyczko Thanks for all the work. Is January 12th the deadline for a version update from upstream, or Feb 12 / Mar 12?

alexmyczko commented 3 years ago

https://buildd.debian.org/status/package.php?p=cadabra2

maybe we can get it to build on more architectures?

kpeeters commented 3 years ago

While in general I am all in favour of getting things to build on as many platforms as possible, the number of users of cadabra on non-amd, non-arm platforms will be zero. I would prefer to spend my already limited time on improving the program itself. (Not to mention that I have no idea where to start with most of these other platforms; I do not have access to any of them, for starters).

alexmyczko commented 3 years ago

@kpeeters have you already tried on the new M1 Apple? Could your provide a very simple non-interactive test, that would be required for a https://github.com/Homebrew/homebrew-core package... let me try on debian gnu/kfreebsd amd64 (can also test on sparc64 if my account still works) let me check...

kpeeters commented 3 years ago

The UK government has not yet started handing out our free M1's, but I'm sure to try once the apple-isation program has reached me. There already is a homebrew package for cadabra2: 

brew tap kpeeters/repo
brew install cadabra2
alexmyczko commented 3 years ago

it's not over yet: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=981534

kpeeters commented 3 years ago

I have pushed and released 2.3.6.6, which removes that one remaining old-style helper package in favour of a new-style one, which hopefully will work fine (just like the others). Can you give this a shot and see if it passes the tests now?

Can do a proper 2.4.0 if that makes life easier, but would like to know first whether it all works now.

alexmyczko commented 3 years ago 
-- Installing: /var/www/debian/cadabra2/2345/cadabra2-2.3.6.6/debian/cadabra2/usr/lib/python3.9/site-packages/cdb/sympy/solvers.cnb
CMake Error at core/packages/cmake_install.cmake:138 (file):
  file INSTALL cannot find
  "/var/www/debian/cadabra2/2345/cadabra2-2.3.6.6/core/packages/cdb/gauge_theory/instantons.cnb":
  No such file or directory.
Call Stack (most recent call first):
  cmake_install.cmake:51 (include)

make[1]: *** [Makefile:141: install] Error 1
make[1]: Leaving directory '/var/www/debian/cadabra2/2345/cadabra2-2.3.6.6/obj-x86_64-linux-gnu'
dh_auto_install: error: cd obj-x86_64-linux-gnu && make -j16 install DESTDIR=/var/www/debian/cadabra2/2345/cadabra2-2.3.6.6/debian/cadabra2 AM_UPDATE_INFO_DIR=no "INSTALL=install --strip-program=true" returned exit code 2
make: *** [debian/rules:14: binary] Error 25
dpkg-buildpackage: error: debian/rules binary subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui failed
kpeeters commented 3 years ago

Dammit, forgot to include that new file. Now fixed. Can you try HEAD again or do I need to do a release?

alexmyczko commented 3 years ago

I can try but just tag a new release then I will save 50% of build time...

kpeeters commented 3 years ago

Tagged as 2.3.6.8.

alexmyczko commented 3 years ago

confirming it's good: https://sid.ethz.ch/debian/cadabra2/2345/

alexmyczko commented 3 years ago

much better https://buildd.debian.org/status/package.php?p=cadabra2