search

23Skidoo / ghc-parmake

A parallel wrapper for 'ghc --make'.
BSD 3-Clause "New" or "Revised" License
10 stars 2 forks source link

ghc-parmake: ghc: runInteractiveProcess: resource exhausted (Resource temporarily unavailable) #1

Open spockz opened 12 years ago

spockz commented 12 years ago

I've made this wrapper for ghc-parmake that only routes --make commands to parmake. I cannot use ghc-parmake as drop-in for GHC as it doesn't report version numbers and such the way GHC does. Maybe there is a loop in this somewhere:

#/usr/bin/env bash
PAR=false;
for thing in "$@"
do
    if [  $thing = "--make"  ]
    then
      PAR=true
      break
    fi
done

if [  $PAR = true  ] 
then
  ghc-parmake -j9 $*
else
  /usr/bin/ghc $*
fi
make INCLUDE_DERIVED_MK=yes EHC_VARIANT=`echo install/101/bin/ehc | sed -n -e 's+install/\([0-9_]*\)/bin/ehc.*+\1+p'` ehc-variant
mkdir -p /tmp/ehc/install-for-build
cd build/libutil/ && \
    ./setup configure --ghc --with-compiler=/Users/alessandro/bin/ghc  --prefix=/tmp/ehc/install-for-build/ --user && \
    ./setup build && \
    ./setup install && \
    echo /tmp/ehc/install-for-build/ins-flg-EH-Util > /tmp/ehc/install-for-build/ins-flg-EH-Util
Configuring EH-Util-1.1...
Preprocessing library EH-Util-1.1...
Building EH-Util-1.1...
ghc-parmake: ghc: runInteractiveProcess: resource exhausted (Resource temporarily unavailable)
make \
       \
      ehc-variant-dflt
mkdir -p /tmp/ehc/install-for-build
cd build/libutil/ && \
    ./setup configure --ghc --with-compiler=/Users/alessandro/bin/ghc  --prefix=/tmp/ehc/install-for-build/ --user && \
    ./setup build && \
    ./setup install && \
    echo /tmp/ehc/install-for-build/ins-flg-EH-Util > /tmp/ehc/install-for-build/ins-flg-EH-Util
Configuring EH-Util-1.1...
Preprocessing library EH-Util-1.1...
Building EH-Util-1.1...
/Users/alessandro/bin/ghc: fork: Resource temporarily unavailable

I've added this code in a file named ghc to my path and it is being found. When compiling packages with cabal build I get this error also. Setting -j to 1 or 2 doesn't help. A similar error happens on the ghc-parmake package as well:

➔ cabal build                                                                                                                              (11:14:29 26-02)
Preprocessing library ghc-parmake-0.1.1...
Preprocessing executables for ghc-parmake-0.1.1...
Preprocessing test suites for ghc-parmake-0.1.1...
Building ghc-parmake-0.1.1...
/Users/alessandro/bin/ghc: fork: Resource temporarily unavailable

GHC: 7.0.3 cabal: 1.10.1.0 ghc-parmake: hackage 0.1.1

23Skidoo commented 12 years ago

Thanks, I'll look into this.

23Skidoo commented 12 years ago

I think I know what happens here. Your ghc wrapper calls ghc-parmake, which in turn calls ghc, which calls ghc-parmake, and so on, ad infinitum. There should be a way to tell ghc-parmake what is the path to the "real" ghc on your system.

spockz commented 12 years ago

Indeed, I thought it had to be something like this. I could maybe check whether the PID of the current process is the PID of the root process? (Will have to write it to somewhere then though.

23Skidoo commented 12 years ago

I've added a --ghc-path option that lets you specify the path to the ghc executable. Append --ghc-path=/usr/bin/ghc to the line that invokes ghc-parmake in your wrapper and check if the issue persists.

spockz commented 12 years ago

Okay, I've pulled your master head: If I leave the check for --make in I get the error below.

#/usr/bin/env bash
PAR=false;
for thing in "$@"
do
    if [  $thing = "--make"  ]
    then
      PAR=true
      break
    fi
done

if [  $PAR = true  ] 
then
  ghc-parmake --ghc-path=/usr/bin/ghc -j9 $*
else
  /usr/bin/ghc $*
fi
➔ cabal configure && cabal build                                                                                                                                 (07:52:05 21-03)
Resolving dependencies...
Configuring ghc-parmake-0.1.1...
Preprocessing library ghc-parmake-0.1.1...
Preprocessing executables for ghc-parmake-0.1.1...
Preprocessing test suites for ghc-parmake-0.1.1...
Building ghc-parmake-0.1.1...
ghc: on the commandline: cannot use `-M' with `--make'
Usage: For basic information, try the `--help' option.

Otherwise when I just wrap ghc-parmake I get this:

➔ cabal configure && cabal build                                                                                                                                 (07:54:50 21-03)
Warning: cannot determine version of /Users/alessandro/bin/ghc :
""
cabal: The program ghc version >=6.4 is required but the version of
/Users/alessandro/bin/ghc could not be determined.
spockz commented 12 years ago

Aha! cabal configure and cabal build seem to work with the following code as the ghc and ghc-parmake wrapper. I can build the ghc-parmake package, as well as install it.

#!/usr/bin/env runhaskell
module Main where

import System  
import System.Exit
import System.Process

import Data.List (intercalate)

main :: IO ()
main = do
  args <- getArgs
  if "--numeric-version" `elem` args then
      system "/usr/bin/ghc --numeric-version" >>= exitWith
    else
      return ()

  if "--make" `elem` args then
      system (intercalate " " $ "/usr/bin/env ghc-parmake" : "-j 1" : "--ghc-path=/usr/bin/ghc" : filter (/="--make") args) >>= exitWith
    else
      rawSystem "/usr/bin/ghc" args >>= exitWith
spockz commented 12 years ago

When compiling the source of the UHC I'm getting the following error, regardless whether I use -j 1 or -j 1+:

EH101/EH/MainAG.hs:75:1:
    Failed to load interface for `EH101.EH.MainAG_common':
      Use -v to see a list of the files searched for.

When compiling the total of the files to parse is shown as 189. When compiling with GHC the total amount of files to compile is 220. So there are missing quite some files.

You can checkout the source of the UHC. Compiling goes as follows:

autoconf
./configure
make -j4 101/ehc;make -j4 101/ehclib EHC_VARIANT_TARGET=js
23Skidoo commented 12 years ago

That's interesting. Maybe the -hidir and -odir directories are not the same? This is not supported yet.

spockz commented 12 years ago

No, I cannot find any trace of that being the case. This is the command that's being issued:

cd build/101/lib-ehc/ && \
    ./setup configure --ghc --with-compiler=/Users/alessandro/bin/ghc  --prefix=/Users/alessandro/Documents/Uni/xp/ehc/install-for-build/ --user && \
    ./setup build && \
    ./setup install && \
        ....
23Skidoo commented 12 years ago

Okay, I'll need to look at this myself. I'll get back to you after I find out what causes this error.

23Skidoo commented 12 years ago

Guess I need to update my GHC version:

[ 54 of 220] Compiling EH101.Pred.CommonCHR ( EH101/Pred/CommonCHR.hs, dist/build/EH101/Pred/CommonCHR.o )
ghc: panic! (the 'impossible' happened)
  (GHC version 7.0.4 for i386-unknown-linux):
    initC: srt_lbl

 Please report this as a GHC bug:  http://www.haskell.org/ghc/reportabug
spockz commented 12 years ago

I am running ghc 7.0.3 on OS X. Which versions do you have of the uulib and the uuagc? I have 0.9.14 and 0.9.40.2, respectively.

23Skidoo commented 12 years ago 
$ cabal info uulib
[...]
    Versions installed: 0.9.14
$ cabal info uuagc
[...]
    Versions installed: 0.9.40.3
spockz commented 12 years ago

I tried with 0.9.40.3 on OS X and it worked as well. I don't know what to do next.

23Skidoo commented 12 years ago

Maybe it's a Linux-only issue. I'll try with GHC 7.4.

23Skidoo commented 12 years ago

Still happens with 7.0.4 and today's checkout. Don't have time to test with 7.4 right now, unfortunately.

spockz commented 12 years ago

I noticed that there are some modules with an _ in the name around where it goes awry. Could that have something to do with it? (BTW: The latest revision also builds with 7.4 now. So that should give you no problems.)

spockz commented 12 years ago

@23Skidoo Could you please send a bug report to uhc-developers@lists.science.uu.nl? It will then be picked up if all is well. :)

23Skidoo commented 12 years ago

Filed a bug report here: http://code.google.com/p/uhc/issues/detail?id=51

spockz commented 12 years ago

@23Skidoo Did you try with GHC 7.4? It looks more like a bug with GHC than the UHC code.

23Skidoo commented 12 years ago

@spockz I tried compiling the version from git master and got the same error. Don't want to migrate to GHC 7.4 yet.

23Skidoo commented 12 years ago

@spockz Compilation proceeds without errors with GHC 7.4.2. Apparently it was a 7.0.4-specific problem. I'll try compiling with ghc-parmake when I have time.