search

bjpop / language-python

A parser for Python 2.x and 3.x written in Haskell
157 stars 46 forks source link

Slow build times in GHC 8.0.x #41

Closed flbulgarelli closed 5 years ago

flbulgarelli commented 6 years ago

Hi!

I am trying add language-python 0.5.4 to an existing project (https://github.com/mumuki/mulang/pull/161), but it is introducing a terrible time penalty in build times. I have also tried to build language-python HEAD from source, with similar results.

It looks like it is not - at least, entirely - caused by language-python, but by GHC >= 8.0 && < 8.4.1 :

I am doing some benchmarks applying https://github.com/antalsz/hs-to-coq/commit/c538e6a92362d22cf6abf45c8f2746ea938ac35a to understand if this is also related to https://ghc.haskell.org/trac/ghc/ticket/14766 as pointed out in https://github.com/simonmar/happy/issues/109#issuecomment-363494692.

I will keep you posted when I have better, stronger evidence of the issue origin, but on the meanwhile, have you tried any workarounds? Thanks!

flbulgarelli commented 6 years ago

Update: by just building Version2.Parser.Parser at HEAD using stack lts-10.2 (ghc 8.2.22) and adding...

{
...
{-# LANGUAGE CPP #-}

#undef __GLASGOW_HASKELL__
#define __GLASGOW_HASKELL__ 709

module Language.Python.Version2.Parser.Parser (parseFileInput, parseSingleInput, parseEval) where
...

...I managed to build it in 6 minutes. Without that hack, Version2.Parser.Parser has not yet finished - 25 minutes and counting.

felixonmars commented 6 years ago

Just to note that the build time is no longer an issue for me since GHC 8.6.1.

bjpop commented 6 years ago

Thanks @felixonmars. I will try it out.

bjpop commented 6 years ago

I tried compiling with ghc 8.6.1 on OS X High Sierra 10.13.6, cabal-install 2.2.0.0, alex 3.2.4 and happy 1.19.9.

I used cabal new-build to build the code.

Unfortunately the compile time and memory usage was still as bad as before. I don't have exact figures, but it took hours to build and memory usage was in the GBs. So it seems I cannot reproduce your success @felixonmars

If possible can you please report versions numbers, OS, and build commands used.

felixonmars commented 6 years ago

It's on Arch Linux x86_64, ghc 8.6.1, alex 3.2.4, happy 1.19.9. The command is:

    runhaskell Setup configure -O --enable-shared --enable-executable-dynamic --disable-library-vanilla \
        --prefix=/usr --docdir="/usr/share/doc/${pkgname}" \
        --dynlibdir=/usr/lib --libsubdir=\$compiler/site-local/\$pkgid \
            -fsplit-base
watashi commented 5 years ago

One more data point that this is still a problem in ghc-8.6.2.

$ uname -a
Linux localhost 4.18.8-arch1-1-ARCH #1 SMP PREEMPT Sat Sep 15 20:34:48 UTC 2018 x86_64 GNU/Linux
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.6.2
$ alex --version
Alex version 3.2.4, (c) 2003 Chris Dornan and Simon Marlow
$ happy --version
Happy Version 1.19.9 Copyright (c) 1993-1996 Andy Gill, Simon Marlow (c) 1997-2005 Simon Marlow
$ cabal --version
cabal-install version 2.4.0.0
compiled using version 2.4.0.1 of the Cabal library
$ pacman -Qo $(which cabal)
/usr/bin/cabal is owned by cabal-static 2.4.0.0-1

cabal build finished after >48 hours.

felixonmars commented 5 years ago

This is weird, I am able to compile language-python 0.5.4 (with merged GHC build patches for 8.2/8.4) within 1 minute. It used to cost at least 8 hours in comparison.

LucianU commented 5 years ago

I'm trying to build from master inside the Docker image provided by FPComplete and I'm hitting the issue with running out of memory.

@felixonmars, in which environment did you build exactly? Or can you tell me the exact steps you took to get a successful build?

LucianU commented 5 years ago

I finally managed to build this in a reasonable time: ~10 minutes.

I ran everything inside a Ubuntu 18.04.1 LTS VM. I also added the lines suggested by @flbulgarelli in https://github.com/bjpop/language-python/issues/41#issuecomment-386885484 to both Language.Python.Version2.Parser.Parser and Language.Python.Version3.Parser.Parser.

felixonmars commented 5 years ago

@LucianU I am building for Arch Linux package, and this is the build script (pure bash): https://git.archlinux.org/svntogit/community.git/tree/trunk/PKGBUILD?h=packages/haskell-language-python

I believe I didn't add the above hack.

LucianU commented 5 years ago

Thanks, but as I already said above, I managed to build it in Ubuntu. Without the CPP thing, it didn't look like it would finish soon.

julian-berbel commented 5 years ago

Hi there! I'm getting the same described behaviour. Build is seemingly endless in GHC 8, and adding the aforementioned hack to files Language.Python.Version2.Parser.Parser and Language.Python.Version3.Parser.Parser takes it down to around 5-10 minutes.

I've opened a fork adding these changes. I know it's not ideal, but since this is really just a GHC bug and there's nothing else to do but wait for it to be fixed, maybe you'll consider merging it in the meanwhile?