search

kpeeters / cadabra2

A field-theory motivated approach to computer algebra.
https://cadabra.science/
GNU General Public License v3.0
223 stars 37 forks source link

building 2.3.5 from source / trying to build a debian package #61

Closed alexmyczko closed 3 years ago

alexmyczko commented 6 years ago

Hi Kasper

Building seems to work, however tests fail. Here's the full log: https://people.phys.ethz.ch/~myczko/debian/cadabra2/

Should I just ignore the failing test, do you have a patch?

kpeeters commented 6 years ago

On which OS/distribution is this? Can you send me the output of ctest -VV > out so I can see why the tests fail? (attach here, or email to info@cadabra.science). Thanks.

alexmyczko commented 6 years ago

$ lsb_release -d Description: Debian GNU/Linux 9.1 (stretch)

$ ctest -VV output link: https://people.phys.ethz.ch/~myczko/debian/cadabra2/ctest-vv.txt

gnaaa how i hate cmake for using absolute path. and being not able to move things around :(

kpeeters commented 6 years ago

That's funny, it complains that it cannot find /scratch/cadabra2/cadabra2-2.1.6/core/cadabra2, but that is the location of a python script which, if you cloned the repo correctly, is executable. Can you do

ls -la /scratch/cadabra2/cadabra2-2.1.6/core/cadabra2

to see what's wrong with that file?

alexmyczko commented 6 years ago

The thing is, i built it in /scratch/cadabra2, later moved it to my home to have it available by web server (the build log etc). There's nothing in /scratch/cadabra2 anymore, so I'll skip the ls -la output. And just rebuild it again, at the new location...

kpeeters commented 6 years ago

If you want 'make test' to work, you need to do that inside the 'build' directory. So clone the repo, then

cd cadabra2
mkdir build
cmake ..
make
make test

If that still fails, send me the output of ctest -VV again (run from the build directory).

alexmyczko commented 6 years ago

I am now skipping testing, and get stuck with install:

... (full log same URL): -- Installing: /home/sengun/public_html/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/share/icons/hicolor/128x128/apps/cadabra2-gtk.png -- Installing: /home/sengun/public_html/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/share/icons/hicolor/256x256/apps/cadabra2-gtk.png -- Installing: /home/sengun/public_html/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/share/icons/hicolor/scalable/apps/cadabra2-gtk.svg -- Installing: /home/sengun/public_html/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/share/applications/cadabra2-gtk.desktop touch: cannot touch '/usr/share/icons/hicolor/icon-theme.cache': Permission denied make[1]: Leaving directory '/home/sengun/public_html/debian/cadabra2/cadabra2-2.1.6/obj-x86_64-linux-gnu' dh_installdocs -O--buildsystem=cmake dh_installchangelogs -O--buildsystem=cmake dh_icons -O--buildsystem=cmake dh_perl -O--buildsystem=cmake dh_usrlocal -O--buildsystem=cmake dh_usrlocal: debian/cadabra2/usr/local/lib/python3.5/dist-packages/cadabra2.so is not a directory dh_usrlocal: debian/cadabra2/usr/local/lib/python3.5/dist-packages/cadabra2_defaults.py is not a directory dh_usrlocal: debian/cadabra2/usr/local/lib/python3.5/dist-packages/cdb/relativity/init.py is not a directory dh_usrlocal: debian/cadabra2/usr/local/lib/python3.5/dist-packages/cdb/relativity/schwarzschild.py is not a directory rmdir: failed to remove 'debian/cadabra2/usr/local/lib/python3.5/dist-packages/cdb/relativity': Directory not empty dh_usrlocal: rmdir debian/cadabra2/usr/local/lib/python3.5/dist-packages/cdb/relativity returned exit code 1 debian/rules:16: recipe for target 'binary' failed make: *** [binary] Error 1 dpkg-buildpackage: error: fakeroot debian/rules binary gave error exit status 2 debuild: fatal error at line 1116: dpkg-buildpackage -rfakeroot -us -uc failed

kpeeters commented 6 years ago

Did you run this with some funky CMAKE_INSTALL_PREFIX by any chance?

alexmyczko commented 6 years ago

Problem is the install target creates /usr/local files: local local/share local/share/texmf local/share/texmf/tex local/share/texmf/tex/latex local/share/texmf/tex/latex/cadabra2 local/share/texmf/tex/latex/cadabra2/tableaux.sty local/lib local/lib/python3.6 local/lib/python3.6/dist-packages local/lib/python3.6/dist-packages/cadabra2_defaults.py local/lib/python3.6/dist-packages/cdb local/lib/python3.6/dist-packages/cdb/gauge_theory local/lib/python3.6/dist-packages/cdb/gauge_theory/instantons.py local/lib/python3.6/dist-packages/cdb/gauge_theory/init.py local/lib/python3.6/dist-packages/cdb/relativity local/lib/python3.6/dist-packages/cdb/relativity/schwarzschild.py local/lib/python3.6/dist-packages/cdb/relativity/init.py local/lib/python3.6/dist-packages/cadabra2.so

which it shouldn't.

Not really, here's what I use: cd obj-i686-linux-gnu && cmake .. -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_BUILD_TYPE=None -DCMAKE_INSTALL_SYSCONFDIR=/etc -DCMAKE_INSTALL_LOCALSTATEDIR=/var -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON

kpeeters commented 6 years ago

What does the cmake stage say about where it will install LaTeX style files? Something like

-- Installing LaTeX style files in /usr/local/share/texmf/tex/latex/cadabra2

Some of the logic that I use to figure out sensible defaults is not appropriate for all systems, but I'll install a VM with Debian 9.1.2 tonight to see what needs fixing.

alexmyczko commented 6 years ago

Yes it does try to install them to /usr/local, which is also the case with your provided deb packages for Ubuntu:

cadabra2# grep usr/local cadabra2_2.1.6-1_i386.build -- Python site path at /usr/local/lib/python3.6/dist-packages -- Installing LaTeX style files in /usr/local/share/texmf/tex/latex/cadabra2 -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cadabra2.so -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cadabra2_defaults.py -- Up-to-date: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cdb/relativity/init.py -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cdb/relativity/schwarzschild.py -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cdb/gauge_theory/init.py -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cdb/gauge_theory/instantons.py -- Up-to-date: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/lib/python3.6/dist-packages/cdb -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/share/texmf/tex/latex/cadabra2 -- Installing: /var/www/debian/cadabra2/cadabra2-2.1.6/debian/cadabra2/usr/local/share/texmf/tex/latex/cadabra2/tableaux.sty texhash: Updating /usr/local/share/texmf/ls-R... dh_usrlocal: debian/cadabra2/usr/local/share/texmf/tex/latex/cadabra2/tableaux.sty is not a directoryrmdir: failed to remove 'debian/cadabra2/usr/local/share/texmf/tex/latex/cadabra2': Directory not empty dh_usrlocal: rmdir debian/cadabra2/usr/local/share/texmf/tex/latex/cadabra2 returned exit code 1

kpeeters commented 6 years ago

Yes, I messed up 2.1.6, hence the new 2.1.7 packages. Those should install in /usr/.

alexmyczko commented 6 years ago

What is the status on having cadabra pre-packaged in debian, should it be kept available (I read you stopped development on it, and even don't provide the sources anymore, the normal way). And can cadabra2 be used (and distributed with debian) instead as replacement, or rather additionally to the existing/old cadabra?

kpeeters commented 6 years ago

Cadabra 1.x notebooks (made with the 'cadabra' package in debian) cannot be read by cadabra 2.x, so it makes sense to keep cadabra 1.x around. The sources for 1.x are at github in kpeeters/cadabra. As far as I know this still builds on recent debian versions (and I will provide fixes for any reported build issues). For all new users, 2.x is the way to go. I'm happy to work with people on getting it in the standard debian repos, but at the moment it is quicker for me to simply make .deb packages available myself.

alexmyczko commented 6 years ago

So I am waiting for 2.1.7 to appear at https://github.com/kpeeters/cadabra2/releases ... :)

kpeeters commented 6 years ago

Done.

alexmyczko commented 6 years ago

Thanks, however that also installs stuff in /usr/local .. anyways I'll take care of it packaging wise, don't bother.

alexmyczko commented 6 years ago

trying with 2.2.0, still the same problem :(

alexmyczko commented 5 years ago

i've got packages of 2.2.4, can you test them? http://phd-sid.ethz.ch/debian/cadabra2/

kpeeters commented 5 years ago

Which debian release is this meant to work on?

kpeeters commented 5 years ago

And if it required any patching, can you please let me know so I can merge that?

alexmyczko commented 5 years ago

for buster need be ready before 12th Feb, will get back tomorrow have to fix more things: patches will be manual pages and the .desktop file...

kpeeters commented 5 years ago

Ok, I will try to do a 2.2.6 release a few days before that as there is some nice new stuff going in right now that would be good to have. Thanks for your efforts, much appreciated!

alexmyczko commented 5 years ago

do you want to write manual pages? shall it co exist with old cadabra or replace it?

kpeeters commented 5 years ago

Make it co-exist please. I will add manual pages.

kpeeters commented 5 years ago

I have added manual pages for cadabra2 and cadabra2-gtk to the master branch, will add a few more for the other build targets as well.

alexmyczko commented 5 years ago

that really helps a lot. thank you

alexmyczko commented 5 years ago

this one doesn't work anymore, kasper.peeters@aei.mpg.de, what's your current email? we got the itp now: https://bugs.debian.org/920238

alexmyczko commented 5 years ago

These are the remaining problems:

E: cadabra2 source: source-is-missing web2/cadabra2/source/static/scripts/hyphenator.js line length is 32481 characters (>512)
E: cadabra2 source: source-is-missing web2/cadabra2/source/static/scripts/jquery.min.js
E: cadabra2: binary-or-shlib-defines-rpath usr/bin/cadabra-server /usr/lib
E: cadabra2: embedded-library usr/bin/cadabra-server: libjsoncpp
E: cadabra2: binary-or-shlib-defines-rpath usr/bin/cadabra2-gtk /usr/lib
E: cadabra2: embedded-library usr/bin/cadabra2-gtk: libjsoncpp
E: cadabra2: binary-or-shlib-defines-rpath usr/bin/cadabra2cadabra /usr/lib
E: cadabra2: embedded-library usr/bin/cadabra2cadabra: libjsoncpp
E: cadabra2: binary-or-shlib-defines-rpath usr/bin/cadabra2html /usr/lib
E: cadabra2: embedded-library usr/bin/cadabra2html: libjsoncpp
E: cadabra2: binary-or-shlib-defines-rpath usr/bin/cadabra2python /usr/lib
E: cadabra2: embedded-library usr/bin/cadabra2python: libjsoncpp
E: cadabra2: arch-dependent-file-in-usr-share usr/share/cadabra2/python/cadabra2.so
E: cadabra2: binary-or-shlib-defines-rpath usr/share/cadabra2/python/cadabra2.so /usr/lib
E: cadabra2: embedded-library usr/share/cadabra2/python/cadabra2.so: libjsoncpp
W: cadabra2: privacy-breach-generic usr/share/cadabra2/notebook.html [<link rel="stylesheet" href="http://cadabra.science/static/fonts/serif/cmun-serif.css">] (http://cadabra.science/static/fonts/serif/cmun-serif.css)
W: cadabra2: binary-without-manpage usr/bin/cadabra-server
E: cadabra2: python-script-but-no-python-dep usr/bin/cadabra2 #!/usr/bin/python3.7
kpeeters commented 5 years ago

email for cadabra purposes: info@cadabra.science

kpeeters commented 5 years ago

The first two refer to the files to build the web site, you don't need to package those. All rpath settings were added after zillions of hours spent on making this build and run on linux, macos and windows; I'm very hesitant to remove them for fear of breaking things again. Is this a showstopper? The embedded libjsoncpp is there for a similar reason: it's the only way to ensure that this builds on all platforms without endless tricks to find the right version of that library. I'd prefer to keep it in, especially since we will probably switch to a header-only JSON parser soon. The 'cadabra2.so' is a python module, which for various reasons we cannot install in a python-specific path because that breaks on various platforms. Am happy to move it somewhere else though if you have a better suggestion. The privacy-breach-generic is there because the HTML export function produces fully self-contained single-file HTML files (to make sharing as easy as possible), and that means including the font-related css entirely in every notebook, or relying on an external source. It's hosted on the master site though. I will make a man page for cadabra-server (it's not really meant to be run by users, but it doesn't take a lot of time to write one either). Finally, I don't understand the python dependency; probably because your packages do not have python listed as a dependency (my debian packages which are produced by cpack do have python listed as dependency).

alexmyczko commented 5 years ago

I can just delete them from the source package, and all works fine?

rpath: is not a showstopper as long as you don't provide a public API to other software

the embedded libjsoncpp is not great, i'll try to use the system libjsoncpp and rip yours away: is https://packages.debian.org/search?keywords=libjsoncpp-dev 1.7.4 good enough?

python: no idea about python, except i can start , import, exit it

can you not add the fonts locally, or depend on them if they are already in a package?

thanks for the man pages, that's a great help.

i'll try about the python deps (tried with python3, maybe just i'll add python) and it works

kpeeters commented 5 years ago

Nobody except me really needs the web2 directory; that's just all the materials for the web site. Feel free to exclude from the source package.

No public API, so let's leave rpath.

You can attempt to build against debian libjsoncpp and that will probably work. I am not going to merge that change into my repo though, given that I'll move away to a more modern JSON parsing library in the near future.

For the fonts: no, if they are stored locally, then people cannot just share the produced HTML file. The idea is that Cadabra can produce a single HTML file which is self-contained and can display the whole notebook on any other computer. That requires some reliance on resources stored online.

Using 'python' instead of 'python3' would suggest to me that there is dependence on Python 2.x, but maybe Debian has changed its naming logic.

alexmyczko commented 5 years ago

web2 dropped, python lintian msgs fixed, waiting for man/man1/cadabra-server.1 and going to fix the embedded libjsoncpp-dev

the font thing i can overcome for debian only by depending on https://packages.debian.org/sid/fonts-cmu

kpeeters commented 5 years ago

man/man1/cadabra-server.1 now present.

alexmyczko commented 5 years ago

Added! Trying to use the fonts-cmu, I'm not able to read your css: Opeing http://cadabra.science/static/fonts/serif/cmun-serif.css I get: Not Found The requested URL /static/fonts/serif/cmun-serif.css was not found on this server.

I'm a bit lost getting rid of your embedded jsoncpp, I can only find some header files:

$ find . -name "*jsoncpp*"
./libs/jsoncpp
./libs/jsoncpp/jsoncpp.cpp
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2cadabra.dir/__/libs/jsoncpp
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2cadabra.dir/__/libs/jsoncpp/jsoncpp.cpp.o
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2.dir/__/libs/jsoncpp
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2.dir/__/libs/jsoncpp/jsoncpp.cpp.o
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2python.dir/__/libs/jsoncpp
./obj-x86_64-linux-gnu/core/CMakeFiles/cadabra2python.dir/__/libs/jsoncpp/jsoncpp.cpp.o
./obj-x86_64-linux-gnu/client_server/CMakeFiles/cadabra-server.dir/__/libs/jsoncpp
./obj-x86_64-linux-gnu/client_server/CMakeFiles/cadabra-server.dir/__/libs/jsoncpp/jsoncpp.cpp.o
./obj-x86_64-linux-gnu/client_server/CMakeFiles/cadabra_client.dir/__/libs/jsoncpp
./obj-x86_64-linux-gnu/client_server/CMakeFiles/cadabra_client.dir/__/libs/jsoncpp/jsoncpp.cpp.o

Could you help me what to edit in cmakefiles to get rid of the embedded one, and use the system one?

alexmyczko commented 5 years ago

otherwise i thought there was a way to embed fonts into css/html: https://github.com/ecomfe/fontmin#css

kpeeters commented 5 years ago

I have pushed changes to master which add an option 'ENABLE_SYSTEM_JSONCPP' to cmake. So you can configure with '-DENABLE_SYSTEM_JSONCPP=ON' and have it build against the jsoncpp provided by Debian.

I can't sort that font issue, sorry; if you are really worried about it just rip that reference to the font line out (though it seems to me that this is no worse than having a link to the web site embedded in the code; I prefer you keep it in).

alexmyczko commented 5 years ago

HALP! i tried that option and I get this: http://phd-sid.ethz.ch/debian/cadabra2/test/cadabra2_2.2.4%2Bgithub20190204-1_amd64.build

kpeeters commented 5 years ago

Sorry, forgot to check in FindJSONCPP.cmake (jsoncpp does not come with a cmake configuration file). Please try again.

alexmyczko commented 5 years ago

That worked! Thank you.

kpeeters commented 5 years ago

I just discovered that I had forgotten to include python3-gmpy2 as a dependency; can you add it please for the Debian package?

kpeeters commented 5 years ago

Ah, it's actually in my packaging.cmake already; you probably copied that. Ignore.

alexmyczko commented 5 years ago

No it was missing in my package, updated: http://phd-sid.ethz.ch/debian/cadabra2/test/

alexmyczko commented 4 years ago

Somehow building seems to work to 100% however something fails, unclear to me what, if you see it? http://phd-sid.ethz.ch/debian/cadabra2/cadabra2_2.2.8-1_amd64.build

kpeeters commented 4 years ago

It's building for linking against the system-supplied libjsoncpp, but then not linking to that library properly. I have pushed a commit to master which fixes this; can you cherry-pick that one and see that it works? If that's too difficult I can do a proper 2.2.9 release, but that will have to wait a few hours.

alexmyczko commented 4 years ago

I'd prefer a 2.2.9 if you don't mind. Happy New Year.

Don't worry, I've taken a checkout, and can confirm: it works!

alexmyczko commented 4 years ago

so many commits, what do you think about a new release?

kpeeters commented 4 years ago

You want to be the release manager? ;-) But seriously, these things take time, of which there is a universal shortage. Yes, it's on my TODO list.

alexmyczko commented 4 years ago

Absoluetly not, take your time as much as you want or need :)

kpeeters commented 4 years ago

There are 2.3.0 (last stable release) and 2.3.1 packages (current github master) for all supported platforms now on the cadabra site. I would like to push a cadabra package into the official debian repos, but lack the time for that; are you willing to help? There is an open issue on the debian tracker, but I haven't about this for a long time.