cschwan / sage-on-gentoo

(Unofficial) Gentoo Overlay for Sage- and Sage-related ebuilds
82 stars 26 forks source link

Sage PDF docs (sci-mathematics/sage with doc-pdf USE flag set) are failing to build correctly: cannot allocate memory #462

Closed fusion809 closed 7 years ago

fusion809 commented 7 years ago

Hi,

sci-mathematics/sage builds fine for most of the build, except near the end when it is trying to build the Sage PDF docs which is when it returns:

[graphs   ] reading sources... [ 31%] sage/graphs/digraph
[repl     ] reading sources... [ 91%] sage/repl/rich_output/output_graphics3d
Error building the documentation.
Traceback (most recent call last):
  File "sage_setup/docbuild/__main__.py", line 2, in <module>
    main()
  File "/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7/sage_setup/docbuild/__init__.py", line 1667, in main
    builder()
  File "/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7/sage_setup/docbuild/__init__.py", line 320, in _wrapper
    getattr(get_builder(document), name)(*args, **kwds)
  File "/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7/sage_setup/docbuild/__init__.py", line 510, in _wrapper
    build_many(build_ref_doc, L)
  File "/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7/sage_setup/docbuild/__init__.py", line 252, in build_many
    ret = x.get(99999)
  File "/usr/lib64/python2.7/multiprocessing/pool.py", line 567, in get
    raise self._value
OSError: [Errno 12] Cannot allocate memory

Note: incremental documentation builds sometimes cause spurious
error messages. To be certain that these are real errors, run
"make doc-clean" first and try again.
 * ERROR: sci-mathematics/sage-7.6::sage-on-gentoo failed (compile phase):
 *   failed to produce pdf doc
 * 
 * Call stack:
 *     ebuild.sh, line  115:  Called src_compile
 *   environment, line 4711:  Called distutils-r1_src_compile
 *   environment, line 1320:  Called _distutils-r1_run_common_phase 'python_compile_all'
 *   environment, line  517:  Called _distutils-r1_run_foreach_impl 'python_compile_all'
 *   environment, line  531:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 3982:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 2976:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 2974:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line  873:  Called distutils-r1_run_phase 'python_compile_all'
 *   environment, line 1308:  Called python_compile_all
 *   environment, line 3541:  Called die
 * The specific snippet of code:
 *           "${PYTHON}" sage_setup/docbuild/__main__.py all pdf || die "failed to produce pdf doc";
 * 
 * If you need support, post the output of `emerge --info '=sci-mathematics/sage-7.6::sage-on-gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sci-mathematics/sage-7.6::sage-on-gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sci-mathematics/sage-7.6/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-mathematics/sage-7.6/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7'
 * S: '/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src'

Now you might wonder how I managed to build sci-mathematics/sage given the sage-latex build fails (per #461), oddly enough after I installed a few extra deps sage-latex is no longer pulled in by sage. emerge -pqv returns:

[ebuild   R   ] sci-mathematics/sage-7.6  USE="X doc-html* doc-pdf* latex* modular_decomposition* sagenb* -bliss -debug -doc-html-bin* -doc-pdf-bin -libbraiding -libhomfly -testsuite" L10N="en -ca -de -fr -hu -it -ja -pt -ru -tr" PYTHON_TARGETS="python2_7"

while emerge --info returns:

Portage 2.3.5 (python 3.4.6-final-0, default/linux/amd64/13.0/desktop/plasma/systemd, gcc-5.4.0, glibc-2.24-r1, 4.9.20-gentoo x86_64)
=================================================================
                         System Settings
=================================================================
System uname: Linux-4.9.20-gentoo-x86_64-Intel-R-_Core-TM-_i7-4700MQ_CPU_@_2.40GHz-with-gentoo-2.3
KiB Mem:    16344584 total,   8884096 free
KiB Swap:          0 total,         0 free
Timestamp of repository gentoo: Wed, 05 Apr 2017 23:00:01 +0000
sh bash 4.4_p12
ld GNU ld (Gentoo 2.27 p1.0) 2.27
app-shells/bash:          4.4_p12::gentoo
dev-java/java-config:     2.2.0-r3::gentoo
dev-lang/perl:            5.24.1-r1::gentoo
dev-lang/python:          2.7.13-r100::sage-on-gentoo, 3.4.6::gentoo
dev-util/cmake:           3.7.2-r1::gentoo
dev-util/pkgconfig:       0.29.2::gentoo
sys-apps/baselayout:      2.3::gentoo
sys-apps/openrc:          0.24.2::gentoo
sys-apps/sandbox:         2.10-r4::gentoo
sys-devel/autoconf:       2.13::gentoo, 2.69-r2::gentoo
sys-devel/automake:       1.13.4-r1::gentoo, 1.15-r2::gentoo
sys-devel/binutils:       2.27::gentoo
sys-devel/gcc:            4.9.4::gentoo, 5.4.0-r3::gentoo
sys-devel/gcc-config:     1.8-r1::gentoo
sys-devel/libtool:        2.4.6-r4::gentoo
sys-devel/make:           4.2.1::gentoo
sys-kernel/linux-headers: 4.9::gentoo (virtual/os-headers)
sys-libs/glibc:           2.24-r1::gentoo
Repositories:

gentoo
    location: /usr/portage
    sync-type: rsync
    sync-uri: rsync://rsync.gentoo.org/gentoo-portage
    priority: -1000

fusion809
    location: /usr/local/fusion809-overlay
    masters: gentoo

flatpak-overlay
    location: /usr/local/flatpak-overlay
    sync-type: git
    sync-uri: git://github.com/fosero/flatpak-overlay.git
    masters: gentoo
    priority: 50

gamerlay
    location: /var/lib/layman/gamerlay
    masters: gentoo
    priority: 50

games-overlay
    location: /var/lib/layman/games-overlay
    masters: gentoo
    priority: 50

grub2-themes
    location: /var/lib/layman/grub2-themes
    masters: gentoo
    priority: 50

hossie
    location: /var/lib/layman/hossie
    masters: gentoo
    priority: 50

sage-on-gentoo
    location: /var/lib/layman/sage-on-gentoo
    masters: gentoo science
    priority: 50

science
    location: /var/lib/layman/science
    masters: gentoo
    priority: 50

spring
    location: /var/lib/layman/spring
    masters: gentoo
    priority: 50

steam-overlay
    location: /var/lib/layman/steam-overlay
    masters: gentoo
    priority: 50

ACCEPT_KEYWORDS="amd64 ~amd64"
ACCEPT_LICENSE="* -@EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc /usr/share/config /usr/share/gnupg/qualified.txt"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
FCFLAGS="-O2 -pipe"
FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr"
FFLAGS="-O2 -pipe"
GENTOO_MIRRORS="http://distfiles.gentoo.org"
LANG="en_AU.UTF-8"
LDFLAGS="-Wl,-O1 -Wl,--as-needed"
MAKEOPTS="-j9"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git"
PORTAGE_TMPDIR="/var/tmp"
USE="X a52 aac acl acpi alsa amd64 berkdb branding bumblebee bzip2 cairo cdda cdr cli cracklib crypt cups cxx dbus declarative dri dts dvd dvdr emboss encode exif fam firefox flac fortran gdbm gif glamor gpm gtk iconv ipv6 jpeg kde kipi kms kwallet lcms ldap libnotify mad mng modules mp3 mp4 mpeg multilib ncurses nls nptl nvidia ogg opengl openmp pam pango pcre pdf phonon plasma png policykit ppds qml qt3support qt4 qt5 readline sdl seccomp semantic-desktop session spell ssl startup-notification svg systemd tcpd tiff truetype udev udisks unicode upower usb vorbis wayland widgets wxwidgets x264 xattr xcb xcomposite xinerama xml xscreensaver xv xvid xwayland zlib" ABI_X86="64" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem ymfpci" APACHE2_MODULES="authn_core authz_core socache_shmcb unixd actions alias auth_basic authn_alias authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache cgi cgid dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation rewrite setenvif speling status unique_id userdir usertrack vhost_alias" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="mmx mmxext sse sse2" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" INPUT_DEVICES="evdev synaptics" KERNEL="linux" L10N="en en_AU en_GB en_US en-AU en-GB" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-console presenter-minimizer" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python2_7" PYTHON_TARGETS="python2_7 python3_4" RUBY_TARGETS="ruby21" USERLAND="GNU" VIDEO_CARDS="nvidia intel" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account"
Unset:  CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON

=================================================================
                        Package Settings
=================================================================

sci-mathematics/sage-7.6::sage-on-gentoo was built with the following:
USE="X doc-html-bin -bliss -debug -doc-html -doc-pdf -doc-pdf-bin -latex -libbraiding -libhomfly -modular_decomposition -sagenb -testsuite" ABI_X86="64" L10N="en -ca -de -fr -hu -it -ja -pt -ru -tr" PYTHON_TARGETS="python2_7"

Thanks for your time, Brenton

fusion809 commented 7 years ago

My build log is here http://paste2.org/74AOb7NM

kiwifb commented 7 years ago

The building of the documentation, pdf in particular is a memory hog. Upstream is aware of it and I cannot do much to make it fit. By curiosity, how much memory do you have available.

This is one of the reason why I provide pre-build documentation with the doc-html-bin and doc-pdf-bin useflags and why this is the default for the html documentation.

fusion809 commented 7 years ago

I have 16 GB RAM available, of all PCs I'd expect mine to manage it.

kiwifb commented 7 years ago

I would have thought it would be sufficient too, but I don't have concrete numbers on that (I have 32GB here). Your /var/tmp/portage` is not in shm is it? That would potentially half your available memory.

fusion809 commented 7 years ago

How would I test if it is? If it's not the default it shouldn't be, most of my Gentoo installation is pretty much as default, except I use systemd instead of OpenRC as my init system.

fusion809 commented 7 years ago

If it should be listed in /etc/fstab (which Google results seem to suggest) then it's not my /etc/fstab is:

# /etc/fstab: static file system information.
#
# noatime turns off atimes for increased performance (atimes normally aren't 
# needed); notail increases performance of ReiserFS (at the expense of storage 
# efficiency).  It's safe to drop the noatime options if you want and to 
# switch between notail / tail freely.
#
# The root filesystem should have a pass number of either 0 or 1.
# All other filesystems should have a pass number of 0 or greater than 1.
#
# See the manpage fstab(5) for more information.
#

# <fs>                  <mountpoint>    <type>          <opts>          <dump/pass>

# NOTE: If your BOOT partition is ReiserFS, add the notail option to opts.
#
# NOTE: Even though we list ext4 as the type here, it will work with ext2/ext3
#       filesystems.  This just tells the kernel to use the ext4 driver.
#
# NOTE: You can use full paths to devices like /dev/sda3, but it is often
#       more reliable to use filesystem labels or UUIDs. See your filesystem
#       documentation for details on setting a label. To obtain the UUID, use
## <fs>          <mountpoint>    <type>      <opts>          <dump/pass>

/dev/sda1       /boot/efi       vfat        noauto,noatime  1 2
/dev/sda2       /ot             ext4        noatime         0 1
/dev/sda4       /               ext4        noatime         0 1
/dev/sdb1       /data           ext4        noatime         0 1
/dev/sda3       /funtoo         ext4        noatime         0 1
kiwifb commented 7 years ago

On systemd you /tmp is on /dev/shm by default but that's not it. After checking carefully your log you are not even building pdf. You are still building html (which matches my understanding better, I didn't think you could build the html and not the pdf).

fusion809 commented 7 years ago

I just tried to recompile this package with the same USE flags and it gave a slightly different error:

Underfull \hbox (badness 10000) in paragraph at lines 33159--33160
\T1/ptm/m/n/10 Bases: \T1/pcr/m/n/10 sage.tensor.modules.free_module_alt_form.F
reeModuleAltForm \T1/ptm/m/n/10 ,
[428] [429] [430] [431] [432] [433] [434]
Overfull \hbox (8.34448pt too wide) in paragraph at lines 33729--33729
[]\T1/ptm/m/it/10 domain\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 33729--33729
\T1/ptm/m/it/10 la-

Overfull \hbox (47.82422pt too wide) in paragraph at lines 33729--33729
\T1/ptm/m/it/10 tex_name=None\T1/pcr/m/n/10 ) 

Underfull \hbox (badness 8113) in paragraph at lines 33773--33780
\T1/ptm/m/n/10 where $\OML/cmm/m/it/10 T[]\OT1/cmr/m/n/10 (\OML/cmm/m/it/10 M\O
T1/cmr/m/n/10 )$ \T1/ptm/m/n/10 stands for the $\OML/cmm/m/it/10 C[]\OT1/cmr/m/
n/10 (\OML/cmm/m/it/10 M\OT1/cmr/m/n/10 )$\T1/ptm/m/n/10 -module of ten-sor fie
lds of type $\OT1/cmr/m/n/10 (\OML/cmm/m/it/10 k; l\OT1/cmr/m/n/10 )$ \T1/ptm/m
/n/10 on $\OML/cmm/m/it/10 M$ \T1/ptm/m/n/10 (cf.
[435] [436] [437] [438] [439] [440] [441] [442] [443]
Underfull \hbox (badness 8170) in paragraph at lines 34469--34469
[]\T1/ptm/m/it/10 frame=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 chart=None\T1/ptm/
m/n/10 , \T1/ptm/m/it/10 sym-bol=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 la-tex_sy
mbol=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 in-dex_labels=None\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 34469--34469
\T1/ptm/m/it/10 in-dex_latex_labels=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 co-or-
di-nate_labels=True\T1/ptm/m/n/10 , \T1/ptm/m/it/10 only_nonzero=True\T1/ptm/m/
n/10 ,
[444] [445] [446] [447] [448] [449] [450] [451] [452] [453] [454]
Chapter 3.

Overfull \hbox (29.99387pt too wide) in paragraph at lines 35284--35284
[]\T1/ptm/m/it/10 vector_field_module\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 35284--35284
\T1/ptm/m/it/10 ture=None\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 35284--35284
\T1/ptm/m/it/10 la-

Overfull \hbox (17.82422pt too wide) in paragraph at lines 35284--35284
\T1/ptm/m/it/10 tex_name=None\T1/pcr/m/n/10 ) 
[455] [456] [457] [458] [459]
Underfull \hbox (badness 10000) in paragraph at lines 35668--35670
[]\T1/ptm/m/n/10 Note that a bet-ter dis-play of the Christof-fel sym-bols is p
ro-vided by the method

Underfull \hbox (badness 10000) in paragraph at lines 35686--35686
[]\T1/ptm/m/it/10 chart=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 sym-bol=None\T1/pt
m/m/n/10 , \T1/ptm/m/it/10 la-tex_symbol=None\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 35686--35686
\T1/ptm/m/it/10 in-dex_labels=None\T1/ptm/m/n/10 , \T1/ptm/m/it/10 in-dex_latex
_labels=None\T1/ptm/m/n/10 ,

Underfull \hbox (badness 10000) in paragraph at lines 35686--35686
\T1/ptm/m/it/10 co-or-di-nate_labels=True\T1/ptm/m/n/10 , \T1/ptm/m/it/10 only_
nonzero=True\T1/ptm/m/n/10 ,
[460] [461] [462] [463] [464] [465] [466] [467] [468]
Underfull \hbox (badness 10000) in paragraph at lines 1--36327
[]\T1/pcr/m/n/10 subdomain  \T1/ptm/m/n/10 -- open sub-set $\OML/cmm/m/it/10 U$
 \T1/ptm/m/n/10 of the met-ric's do-main (must be an in-stance of
[469] [470] [471] [472] [473] [474] [475 * The ebuild phase 'compile' with pid 12639 appears to have left an orphan
 * process running in the background.

>>> Failed to emerge sci-mathematics/sage-7.6, Log file:

>>>  '/var/tmp/portage/sci-mathematics/sage-7.6/temp/build.log'

 * Messages for package sci-mathematics/sage-7.6:

 * The ebuild phase 'compile' with pid 12639 appears to have left an orphan
 * process running in the background.
 * ERROR: sci-mathematics/sage-7.6::sage-on-gentoo failed (compile phase):
 *   failed to produce pdf doc
 * 
 * Call stack:
 *     ebuild.sh, line  115:  Called src_compile
 *   environment, line 4711:  Called distutils-r1_src_compile
 *   environment, line 1320:  Called _distutils-r1_run_common_phase 'python_compile_all'
 *   environment, line  517:  Called _distutils-r1_run_foreach_impl 'python_compile_all'
 *   environment, line  531:  Called python_foreach_impl 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 3982:  Called multibuild_foreach_variant '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 2976:  Called _multibuild_run '_python_multibuild_wrapper' 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line 2974:  Called _python_multibuild_wrapper 'distutils-r1_run_phase' 'python_compile_all'
 *   environment, line  873:  Called distutils-r1_run_phase 'python_compile_all'
 *   environment, line 1308:  Called python_compile_all
 *   environment, line 3541:  Called die
 * The specific snippet of code:
 *           "${PYTHON}" sage_setup/docbuild/__main__.py all pdf || die "failed to produce pdf doc";
 * 
 * If you need support, post the output of `emerge --info '=sci-mathematics/sage-7.6::sage-on-gentoo'`,
 * the complete build log and the output of `emerge -pqv '=sci-mathematics/sage-7.6::sage-on-gentoo'`.
 * The complete build log is located at '/var/tmp/portage/sci-mathematics/sage-7.6/temp/build.log'.
 * The ebuild environment file is located at '/var/tmp/portage/sci-mathematics/sage-7.6/temp/environment'.
 * Working directory: '/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src-python2_7'
 * S: '/var/tmp/portage/sci-mathematics/sage-7.6/work/sage-7.6/src'
kiwifb commented 7 years ago

OK reviewing your flags from the first post, there must be an error in the ebuild you shouldn't be building this. What's more you are building the html doc in the pdf phase. So something must be calling the building of the pdf but not html.

Have to leave you hanging for the next hour or so.

fusion809 commented 7 years ago

Well I've decided 'til you have a solution, at least, to just do everything the easy way and emerge the package with the binary docs.

kiwifb commented 7 years ago

OK one of the thing you quoted didn't match your build log. So I thought there was something wrong with the ebuild but no. In this instance I see your MAKEOPTS is -j9 your amount of memory per tasks is probably too small. Could you try at -j4 next time?

fusion809 commented 7 years ago

OK, giving your suggestion a go.

fusion809 commented 7 years ago

It worked! Thanks, closing.