snowleopard
commented
6 years ago This looks similar to https://mail.haskell.org/pipermail/ghc-devs/2018-March/015425.html.
Are you sure you've updated submodules?
Closed izgzhen closed 6 years ago
snowleopard
commented
6 years ago This looks similar to https://mail.haskell.org/pipermail/ghc-devs/2018-March/015425.html.
Are you sure you've updated submodules?
izgzhen
commented
6 years ago Are you sure you've updated submodules?
I supposed "yes", but I tried again from scratch anyway:
| Fix _build/stage1/libraries/integer-gmp/package-data.mk
shakeArgsWith 0.000s 0%
Function shake 0.135s 0%
Database read 0.036s 0%
With database 0.000s 0%
Running rules 694.007s 99% =========================
Total 694.178s 100%
Error when running Shake build system:
* _build/stage1/compiler/setup-config
* _build/stage1/compiler/package-data.mk _build/stage1/compiler/setup-config
* _build/stage1/gmp/include/ghc-gmp.h
* libraries/integer-gmp/gmp/config.mk
* libraries/integer-gmp/gmp/config.mk libraries/integer-gmp/integer-gmp.buildinfo
Error, &%> rule failed to produce 2 files (out of 2)
libraries/integer-gmp/gmp/config.mk - MISSING
libraries/integer-gmp/integer-gmp.buildinfo - MISSING
CallStack (from HasCallStack):
error, called at src/Development/Shake/Internal/Rules/Files.hs:209:13 in shake-0.16.3-Gjbjlakw6DJDzoxGYURGp9:Development.Shake.Internal.Rules.Filesagain error.
izgzhen
commented
6 years ago I guess the above error is related to https://github.com/snowleopard/hadrian/blob/9eef6a7c7cce150f39740beb8570f4cbc38b947e/src/Rules/Gmp.hs#L80-L82
snowleopard
commented
6 years ago @izgzhen Note: this is a different error! Now it is consistent with what we have in CI.
Thanks for recording this: I'm unlikely to have time to look into this during this week, but if anyone would like to fix Hadrian (presumably this failure is caused by some changes in GHC), I'm happy to review a PR.
izgzhen
commented
6 years ago this problem seems to be resolved by:
diff --git a/src/Rules/Gmp.hs b/src/Rules/Gmp.hs
index 46fad8a..4dbbfff 100644
--- a/src/Rules/Gmp.hs
+++ b/src/Rules/Gmp.hs
@@ -78,8 +78,7 @@ gmpRules = do
-- This causes integerGmp package to be configured, hence creating the files
[gmpBase -/- "config.mk", gmpBuildInfoPath] &%> \_ -> do
- dataFile <- pkgDataFile gmpContext
- need [dataFile]
+ build $ target gmpContext (Configure $ pkgPath integerGmp) [gmpBuildInfoPath <.> "in"] [gmpBuildInfoPath]
-- Run GMP's configure script
-- TODO: Get rid of hard-coded @gmp@.But there is more problem coming:
Run Ghc LinkHs Stage1: _build/stage1/libraries/integer-gmp/c/cbits/wrappers.dyn_o (and 5 more) => _build/stage1/libraries/integer-gmp/libHSinteger-gmp-1.0.1.0-ghc8.5.20180304.so
Warning: -rtsopts and -with-rtsopts have no effect with -shared.
Call hs_init_ghc() from your main() function to set these options.
configure: error: internal error
shakeArgsWith 0.000s 0%
Function shake 0.137s 0%
Database read 0.301s 0%
With database 0.009s 0%
Running rules 47.886s 99% =========================
Total 48.334s 100%
Error when running Shake build system:
* _build/stage1/libraries/base/setup-config
* _build/stage1/libraries/base/package-data.mk _build/stage1/libraries/base/setup-config
user error (Development.Shake.cmd, system command failed
Command: _build/stage0/utils/ghc-cabal/ghc-cabal configure libraries/base /home/zhen/repos/ghc/_build/stage1/libraries/base --with-ghc=/home/zhen/repos/ghc/inplace/bin/ghc-stage1 --with-ghc-pkg=/home/zhen/repos/ghc/inplace/bin/ghc-pkg --enable-library-vanilla --enable-library-for-ghci --enable-library-profiling --enable-shared '--configure-option=CFLAGS=-fno-stack-protector -I/home/zhen/repos/ghc/_build/generated' --configure-option=LDFLAGS=-fuse-ld=gold --configure-option=CPPFLAGS=-I_build/generated '--gcc-options=-fno-stack-protector -I/home/zhen/repos/ghc/_build/generated -fuse-ld=gold' --configure-option=--with-cc=/usr/bin/gcc --with-gcc=/usr/bin/gcc --with-ld=/usr/bin/ld.gold --with-ar=/usr/bin/ar --with-alex=/usr/local/bin/alex --with-happy=/usr/local/bin/happy -v0 --configure-option=--quiet --configure-option=--disable-option-checking --flags=integer-gmp
Exit code: 1
Stderr:
configure: error: internal error
)this internal error msg can be reproduced by running ./configure under ghc/libraries/base.
chitrak7
commented
6 years ago @izgzhen @snowleopard the error was coming because "struct MD5Context" was not found. I tried to resolve this by removing and recloning the base repository. This resolved the immediate problem but gave rise to multiple new ones.
snowleopard
commented
6 years ago I think this is a good (and urgent!) issue for a beginner. Potential GSoC applicants, give it a try!
chitrak7
commented
6 years ago The error is coming due to the inclusion of this line in libraries/base/configure.ac::240.
AC_CHECK_TYPE([struct MD5Context], [], [AC_MSG_ERROR([internal error])], [#include "include/md5.h"]).
But the file "include/md5.h" is present and is containing struct MD5Context. I have seen the earlier implementations. Earlier implementation returned with value "no" but new implementation causes an error.
snowleopard
commented
6 years ago @chitrak7 How does the Make build system deal with this? Why does it succeed? Presumably, it calls the configure in base somehow differently?
chitrak7
commented
6 years ago I have been trying to figure it out. There are no recent changes in any of the concerned makefiles, but I believe that we have been doing this step wrong since the start. Just we were not able to detect it as it didn't cause any errors. Can you give any pointers on where to look @snowleopard
chitrak7
commented
6 years ago @snowleopard The make system is somehow copying a few c libraries from ghc/includes to ghc/libraries/base/include. This is because md5.h has some headers files which are not present with it, but are present in ghc/includes. After this, ./configure runs fine. But the problem is still persisting.
snowleopard
commented
6 years ago @chitrak7 Aha, that's a very useful piece of information! Can you link to these specific lines in Makefiles?
chitrak7
commented
6 years ago @snowleopard I have not been able to find this in the Makefile. I am deducing this because the file ghc/libraries/base/include/md5.h has a lot of header files such as "HsFFi.h" which are required for configure to compile. These are present in ghc/includes. Hence, some copying must be done.
chitrak7
commented
6 years ago "inplace/bin/ghc-cabal" configure libraries/base dist-install --with-ghc="/home/chitrak/ghc/inplace/bin/ghc-stage1" --with-ghc-pkg="/home/chitrak/ghc/inplace/bin/ghc-pkg" --flags=integer-gmp --disable-library-for-ghci --enable-library-vanilla --enable-library-for-ghci --enable-library-profiling --enable-shared --with-hscolour="/usr/bin/HsColour" --configure-option=CFLAGS="-Wall -fno-stack-protector -Werror=unused-but-set-variable -Wno-error=inline" --configure-option=LDFLAGS=" " --configure-option=CPPFLAGS=" " --gcc-options="-Wall -fno-stack-protector -Werror=unused-but-set-variable -Wno-error=inline " --with-gcc="gcc" --with-ld="ld.gold" --with-ar="ar" --with-alex="/usr/bin/alex" --with-happy="/usr/bin/happy" @snowleopard the configure command used for base.
chitrak7
commented
6 years ago _build/stage0/utils/ghc-cabal/ghc-cabal configure libraries/base /home/chitrak/ghc/_build/stage1/libraries/base --with-ghc=/home/chitrak/ghc/inplace/bin/ghc-stage1 --with-ghc-pkg=/home/chitrak/ghc/inplace/bin/ghc-pkg --enable-library-vanilla --enable-library-for-ghci --enable-library-profiling --enable-shared '--configure-option=CFLAGS=-fno-stack-protector -I/home/chitrak/ghc/_build/generated' --configure-option=LDFLAGS=-fuse-ld=gold --configure-option=CPPFLAGS=-I_build/generated '--gcc-options=-fno-stack-protector -I/home/chitrak/ghc/_build/generated -fuse-ld=gold' --configure-option=--with-cc=/usr/bin/gcc --with-gcc=/usr/bin/gcc --with-ld=/usr/bin/ld.gold --with-ar=/usr/bin/ar --with-alex=/usr/bin/alex --with-happy=/usr/bin/happy --flags=integer-gmp @snowleopard the configure command used by hadrian
chitrak7
commented
6 years ago @snowleopard It looks like the
snowleopard
commented
6 years ago @chitrak7 Hadrian takes a different approach than Make in this respect. Make builds GHC right in the source tree, polluting it with build artefacts. Hadrian tries to put everything into the _build directory. Hadrian is still not perfect, but I think after @angerman's huge PR #445 is merged we'll have no more pollution of the source tree.
Note: this issue might still be related to what you've found. But, changing _build/stage1/packagePath to packagePath/distdir is not a good fix, Hadrian is supposed to produce stuff in _build.
chitrak7
commented
6 years ago @snowleopard There is an interesting error in config.log: configure:28447: checking for struct MD5Context configure:28447: /usr/bin/gcc -c -I../.. -I../../../../includes -fno-stack-protector -I/home/chitrak/ghc/_build/generated -I_build/generated conftest.c >&5 conftest.c:216:10: fatal error: include/md5.h: No such file or directory
^~~~~~~~~~~~~~~~~compilation terminated. The file does exist here.
snowleopard
commented
6 years ago @chitrak7 Thanks for investigating further! I've made a PR with a simple fix that works for me locally: #526. Let's see if it also works for you and our CI instances.
Note that my build still fails, but later:
Command: C:/msys/home/nam83/ghc/inplace/mingw/bin/gcc.exe -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/base -Ilibraries/base/include -Ilibraries/base/C:/msys/home/nam83/ghc/_build/stage1/libraries/base/build/include -IC:/msys/home/nam83/ghc/_build/stage1/rts -IC:/msys/home/nam83/ghc/includes -IC:/msys/home/nam83/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/base/c/cbits/Win32Utils.o.d -MT _build/stage1/libraries/base/c/cbits/Win32Utils.o -x c libraries/base/cbits/Win32Utils.c
Exit code: 1
Stderr:
In file included from libraries/base/cbits/Win32Utils.c:9:0:
libraries/base/include/HsBase.h:294:2: error: #error at least ftruncate or _chsize functions are required to build
#error at least ftruncate or _chsize functions are required to build
^~~~~@snowleopard It is because we are not including package directory in cflags. I think its a typo. The C file looks for a constatnt HAVE_UNISTD_H and HAVE_FTRUNCATE. This constant is stored in config log. so we have to include config.log along with this.
I tried to include library -I_build/stage1/libraries/base/build which didn't work. But including -Ilibraries/base did. both these directories contain the same config.log, but one is generated manually by me and other by hadrian.
chitrak7
commented
6 years ago 1) Error in issue #523 Reason: The file config.log defines a few variables like HAVE_USISTD_H, HAVE_FTRUNCATE. The file HsBase.h uses these definitions to check for ftruncate or _chsize. But this file is generated in directory path/build. Hence we have to include this directory as well.
2) A discrepancy in INCLUDE_DIRS in package-data.mk The file package-data.mk is supposed to give relative paths in category "INCLUDE_DIRS" while compiling a c file. This is evident from the code:
pure [ "-I" ++ pkgPath pkg -/- dir | dir <- incDirs ] But here is how directories are listed in the file:
include /home/chitrak/ghc/_build/stage1/libraries/base/build/include Here, "include" is relative while "/home/chitrak/ghc/_build/stage1/libraries/base/build/include" is absolute.
Now let us look at how the command to build a cfile SetEnv.c looks like:
/usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/base -I_build/stage1/libraries/basebuild -Ilibraries/base/include -Ilibraries/base//home/chitrak/ghc/_build/stage1/libraries/base/build/include -I/home/chitrak/ghc/libraries/integer-gmp/include -I/home/chitrak/ghc/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/rts -I/home/chitrak/ghc/includes -I/home/chitrak/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/base/c/cbits/SetEnv.p_o.d -MT _build/stage1/libraries/base/c/cbits/SetEnv.o -x c libraries/base/cbits/SetEnv.c
I will point out two key errors in this command: 1 > -I_build/stage1/libraries/base/build :: This flag is not there, hence this directory is not included and we cannot find the config.log. 2 > -Ilibraries/base//home/chitrak/ghc/_build/stage1/libraries/base/build/include :: This is supposed to be just /home/chitrak/ghc/_build/stage1/libraries/base/build/include but now it points to an non-existant directory. @izgzhen @snowleopard
chitrak7
commented
6 years ago After changing a few lines of code, my code passed for libraries/base. But on processing libraries/process, same error came. On exploring further, I found that the dependencies in package-data.mk are not correct. Here are the dependency directories: DEP_INCLUDE_DIRS_SINGLE_QUOTED = '/home/chitrak/ghc/libraries/directory/.' '/home/chitrak/ghc/_build/stage1/libraries/directory/.' '/home/chitrak/ghc/libraries/unix/include' '/home/chitrak/ghc/libraries/unix/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/libraries/unix/include' '/home/chitrak/ghc/_build/stage1/libraries/unix/dist-install/build/include' '/home/chitrak/ghc/libraries/time/lib/include' '/home/chitrak/ghc/libraries/time/lib/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/libraries/time/lib/include' '/home/chitrak/ghc/_build/stage1/libraries/time/lib/dist-install/build/include' '/home/chitrak/ghc/libraries/bytestring/include' '/home/chitrak/ghc/libraries/bytestring/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/libraries/bytestring/include' '/home/chitrak/ghc/_build/stage1/libraries/bytestring/dist-install/build/include' '/home/chitrak/ghc/libraries/base/include' '/home/chitrak/ghc/libraries/base/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/libraries/base/include' '/home/chitrak/ghc/_build/stage1/libraries/base/dist-install/build/include' '/home/chitrak/ghc/libraries/integer-gmp/include' '/home/chitrak/ghc/libraries/integer-gmp/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/include' '/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include' '/home/chitrak/ghc/_build/stage1/rts' '/home/chitrak/ghc/includes' '/home/chitrak/ghc/_build/generated'
Clearly, in all instances that use dist-install, we will run into an error if their file will be used. For example: '/home/chitrak/ghc/_build/stage1/libraries/base/include' '/home/chitrak/ghc/_build/stage1/libraries/base/dist-install/build/include' Both of the above two directories do not exist and should instead be '/home/chitrak/ghc/_build/stage1/libraries/base/build/include'
chitrak7
commented
6 years ago Continued.. This belief is further solidified by the fact that the command : /usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/process -I_build/stage1/libraries/process/build -Ilibraries/process/include -Ilibraries/process//home/chitrak/ghc/_build/stage1/libraries/process/build/include -Iinclude -I/home/chitrak/ghc/_build/stage1/libraries/process/build/include -I/home/chitrak/ghc/libraries/directory -I/home/chitrak/ghc/_build/stage1/libraries/directory -I/home/chitrak/ghc/libraries/unix/include -I/home/chitrak/ghc/libraries/unix/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/unix/include -I/home/chitrak/ghc/_build/stage1/libraries/unix/dist-install/build/include -I/home/chitrak/ghc/libraries/time/lib/include -I/home/chitrak/ghc/libraries/time/lib/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/time/lib/include -I/home/chitrak/ghc/_build/stage1/libraries/time/lib/dist-install/build/include -I/home/chitrak/ghc/libraries/bytestring/include -I/home/chitrak/ghc/libraries/bytestring/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/bytestring/include -I/home/chitrak/ghc/_build/stage1/libraries/bytestring/dist-install/build/include -I/home/chitrak/ghc/libraries/base/include -I/home/chitrak/ghc/libraries/base/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/base/include -I/home/chitrak/ghc/_build/stage1/libraries/base/dist-install/build/include -I/home/chitrak/ghc/libraries/integer-gmp/include -I/home/chitrak/ghc/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/rts -I/home/chitrak/ghc/includes -I/home/chitrak/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/process/c/cbits/runProcess.o.d -MT _build/stage1/libraries/process/c/cbits/runProcess.o -x c libraries/process/cbits/runProcess.c Gives error: at least ftruncate or _chsize functions are required to build
But this command works without error: sudo /usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/process -I_build/stage1/libraries/process/build -Ilibraries/process/include -Ilibraries/process/ -I/home/chitrak/ghc/_build/stage1/libraries/process/build/include -Iinclude -I/home/chitrak/ghc/_build/stage1/libraries/process/build/include -I/home/chitrak/ghc/libraries/directory -I/home/chitrak/ghc/_build/stage1/libraries/directory -I/home/chitrak/ghc/libraries/unix/include -I/home/chitrak/ghc/libraries/unix/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/unix/include -I/home/chitrak/ghc/_build/stage1/libraries/unix/dist-install/build/include -I/home/chitrak/ghc/libraries/time/lib/include -I/home/chitrak/ghc/libraries/time/lib/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/time/lib/include -I/home/chitrak/ghc/_build/stage1/libraries/time/lib/dist-install/build/include -I/home/chitrak/ghc/libraries/bytestring/include -I/home/chitrak/ghc/libraries/bytestring/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/bytestring/include -I/home/chitrak/ghc/_build/stage1/libraries/bytestring/dist-install/build/include -I/home/chitrak/ghc/libraries/base/include -I/home/chitrak/ghc/_build/stage1/libraries/base/build -I/home/chitrak/ghc/_build/stage1/libraries/base/build/include -I/home/chitrak/ghc/libraries/integer-gmp/include -I/home/chitrak/ghc/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/include -I/home/chitrak/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/chitrak/ghc/_build/stage1/rts -I/home/chitrak/ghc/includes -I/home/chitrak/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/process/c/cbits/runProcess.o.d -MT _build/stage1/libraries/process/c/cbits/runProcess.o -x c libraries/process/cbits/runProcess.c
The changes in the above two commands are: Removed directories : '/home/chitrak/ghc/_build/stage1/libraries/base/include' '/home/chitrak/ghc/_build/stage1/libraries/base/dist-install/build/include' Added directories : '/home/chitrak/ghc/_build/stage1/libraries/base/build/include', '/home/chitrak/ghc/_build/stage1/libraries/base/build'
Can anyone please check into this.
chitrak7
commented
6 years ago Changes done by me to pass for libraries/base.
diff --git a/src/Settings/Builders/Common.hs b/src/Settings/Builders/Common.hs
index bfcddeb..a8b7fdf 100644
--- a/src/Settings/Builders/Common.hs
+++ b/src/Settings/Builders/Common.hs
@@ -29,7 +29,9 @@ cIncludeArgs = do
mconcat [ not (cross && compilerOrGhc) ? arg "-Iincludes"
, arg $ "-I" ++ root -/- generatedDir
, arg $ "-I" ++ path
+ , arg $ "-I" ++ path -/- "build"
, pure [ "-I" ++ pkgPath pkg -/- dir | dir <- incDirs ]
+ , pure [ "-I" ++ dir | dir <- incDirs ]
, pure [ "-I" ++ unifyPath dir | dir <- depDirs ] ]@izgzhen @snowleopard
izgzhen
commented
6 years ago | Run Ghc FindHsDependencies Stage1: libraries/xhtml/Text/XHtml.hs (and 14 more) => _build/stage1/libraries/xhtml/.dependencies.mk
| Remove file _build/stage1/libraries/xhtml/.dependencies.mk.bak
| Run Hsc2Hs: libraries/base/System/Environment/ExecutablePath.hsc => _build/stage1/libraries/base/System/Environment/ExecutablePath.hs
| Run Hsc2Hs: libraries/base/System/Environment/Blank.hsc => _build/stage1/libraries/base/System/Environment/Blank.hs
Blank.hsc:84:26: fatal error: HsBaseConfig.h: No such file or directory
compilation terminated.
compiling _build/stage1/libraries/base/System/Environment/Blank_hsc_make.c failed (exit code 1)
command was: /usr/bin/gcc -c _build/stage1/libraries/base/System/Environment/Blank_hsc_make.c -o _build/stage1/libraries/base/System/Environment/Blank_hsc_make.o -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/base -Ilibraries/base/include -Ilibraries/base//home/zhen/repos/ghc/_build/stage1/libraries/base/build/include -I/home/zhen/repos/ghc/libraries/integer-gmp/include -I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -DOPTIMISE_INTEGER_GCD_LCM -Wall -Werror=unused-but-set-variable -Wno-error=inline -include _build/stage1/libraries/base/build/autogen/cabal_macros.h -Dx86_64_HOST_ARCH=1 -Dlinux_HOST_OS=1 -D__GLASGOW_HASKELL__=805
shakeArgsWith 0.000s 0%
Function shake 0.142s 0%
Database read 0.175s 0%
With database 0.010s 0%
Running rules 19.315s 98% =========================
Total 19.642s 100%
Error when running Shake build system:
* _build/stage1/libraries/xhtml/libHSxhtml-3000.2.2_p.a
* _build/stage1/libraries/xhtml/Text/XHtml/Internals.p_o
* _build/stage1/libraries/xhtml/Text/XHtml/Internals.p_o _build/stage1/libraries/xhtml/Text/XHtml/Internals.p_hi
* _build/stage1/libraries/base/Data/Char.p_hi
* _build/stage1/libraries/base/Data/Char.p_o _build/stage1/libraries/base/Data/Char.p_hi
* OracleQ (KeyValues ("_build/stage1/libraries/base/.dependencies","_build/stage1/libraries/base/Data/Char.p_o"))
* _build/stage1/libraries/base/.dependencies
* _build/stage1/libraries/base/System/Environment/Blank.hs
user error (Development.Shake.cmd, system command failed
Command: _build/stage0/utils/hsc2hs/hsc2hs --cc=/usr/bin/gcc --ld=/usr/bin/gcc --cross-safe --cflag=-fno-stack-protector --cflag=-Iincludes --cflag=-I_build/generated --cflag=-I_build/stage1/libraries/base --cflag=-Ilibraries/base/include --cflag=-Ilibraries/base//home/zhen/repos/ghc/_build/stage1/libraries/base/build/include --cflag=-I/home/zhen/repos/ghc/libraries/integer-gmp/include --cflag=-I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include --cflag=-I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include --cflag=-I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include --cflag=-I/home/zhen/repos/ghc/_build/stage1/rts --cflag=-I/home/zhen/repos/ghc/includes --cflag=-I/home/zhen/repos/ghc/_build/generated --cflag=-DOPTIMISE_INTEGER_GCD_LCM --cflag=-Wall --cflag=-Werror=unused-but-set-variable --cflag=-Wno-error=inline --cflag=-include --cflag=_build/stage1/libraries/base/build/autogen/cabal_macros.h --lflag=-fuse-ld=gold --cflag=-Dx86_64_HOST_ARCH=1 --cflag=-Dlinux_HOST_OS=1 --cflag=-D__GLASGOW_HASKELL__=805 --template=/home/zhen/repos/ghc/inplace/lib/template-hsc.h libraries/base/System/Environment/Blank.hsc -o _build/stage1/libraries/base/System/Environment/Blank.hs
Exit code: 1
Stderr:
Blank.hsc:84:26: fatal error: HsBaseConfig.h: No such file or directory
compilation terminated.
compiling _build/stage1/libraries/base/System/Environment/Blank_hsc_make.c failed (exit code 1)
command was: /usr/bin/gcc -c _build/stage1/libraries/base/System/Environment/Blank_hsc_make.c -o _build/stage1/libraries/base/System/Environment/Blank_hsc_make.o -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/base -Ilibraries/base/include -Ilibraries/base//home/zhen/repos/ghc/_build/stage1/libraries/base/build/include -I/home/zhen/repos/ghc/libraries/integer-gmp/include -I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -DOPTIMISE_INTEGER_GCD_LCM -Wall -Werror=unused-but-set-variable -Wno-error=inline -include _build/stage1/libraries/base/build/autogen/cabal_macros.h -Dx86_64_HOST_ARCH=1 -Dlinux_HOST_OS=1 -D__GLASGOW_HASKELL__=805
)@chitrak7 I would like to help you but when I tried to build the latest source with ./build -j, the error above is thrown. I will clean up and try again though.
chitrak7
commented
6 years ago Hi @izgzhen. Can you please run this command instead: /usr/bin/gcc -c _build/stage1/libraries/base/System/Environment/Blank_hsc_make.c -o _build/stage1/libraries/base/System/Environment/Blank_hsc_make.o -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/base -Ilibraries/base/include -Ilibraries/base/ -I/home/zhen/repos/ghc/_build/stage1/libraries/base/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/base/build -I/home/zhen/repos/ghc/libraries/integer-gmp/include -I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -DOPTIMISE_INTEGER_GCD_LCM -Wall -Werror=unused-but-set-variable -Wno-error=inline -include _build/stage1/libraries/base/build/autogen/cabal_macros.h -Dx86_64_HOST_ARCH=1 -Dlinux_HOST_OS=1 -D__GLASGOW_HASKELL__=805
izgzhen
commented
6 years ago @chitrak7 thanks for your reply! I just hadrian/build.sh clean it before noticing your comment, and now the build is stuck similarly but for a different command:
| Run Ghc FindHsDependencies Stage1: libraries/integer-gmp/src/GHC/Integer.hs (and 4 more) => _build/stage1/libraries/integer-gmp/.dependencies.mk
libraries/integer-gmp/src/GHC/Integer/Type.hs:28:0: error:
fatal error: HsIntegerGmp.h: No such file or directory
#include "HsIntegerGmp.h"
^
|
28 | #include "HsIntegerGmp.h"
| ^
compilation terminated.
`gcc' failed in phase `C pre-processor'. (Exit code: 1)
shakeArgsWith 0.000s 0%
Function shake 0.133s 1%
Database read 0.200s 2%
With database 0.010s 0%
Running rules 7.472s 95% =========================
Total 7.816s 100%
Error when running Shake build system:
* _build/stage1/libraries/integer-gmp/libHSinteger-gmp-1.0.1.0.a
* _build/stage1/libraries/integer-gmp/GHC/Integer/Type.o
* _build/stage1/libraries/integer-gmp/GHC/Integer/Type.o _build/stage1/libraries/integer-gmp/GHC/Integer/Type.hi
* OracleQ (KeyValues ("_build/stage1/libraries/integer-gmp/.dependencies","_build/stage1/libraries/integer-gmp/GHC/Integer/Type.o"))
* _build/stage1/libraries/integer-gmp/.dependencies
user error (Development.Shake.cmd, system command failed
Command: inplace/bin/ghc-stage1 -M -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id integer-gmp-1.0.1.0' '-package-id ghc-prim-0.5.2.0' -i -i_build/stage1/libraries/integer-gmp -i_build/stage1/libraries/integer-gmp/build/autogen -ilibraries/integer-gmp/src/ -Iincludes -I_build/generated -I_build/stage1/libraries/integer-gmp -Ilibraries/integer-gmp/include -Ilibraries/integer-gmp//home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -I_build/generated -optc-I_build/generated -optP-include -optP_build/stage1/libraries/integer-gmp/build/autogen/cabal_macros.h -optc-fno-stack-protector -odir _build/stage1/libraries/integer-gmp -hidir _build/stage1/libraries/integer-gmp -stubdir _build/stage1/libraries/integer-gmp -include-pkg-deps -dep-makefile _build/stage1/libraries/integer-gmp/.dependencies.mk -dep-suffix '' -dep-suffix p_ -dep-suffix dyn_ libraries/integer-gmp/src/GHC/Integer.hs libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs libraries/integer-gmp/src/GHC/Integer/Logarithms.hs libraries/integer-gmp/src/GHC/Integer/Logarithms/Internals.hs libraries/integer-gmp/src/GHC/Integer/Type.hs -O2 -H32m -this-unit-id integer-gmp -Wall -XHaskell2010 -Wno-deprecated-flags
Exit code: 1
Stderr:
libraries/integer-gmp/src/GHC/Integer/Type.hs:28:0: error:
fatal error: HsIntegerGmp.h: No such file or directory
#include "HsIntegerGmp.h"
^
|
28 | #include "HsIntegerGmp.h"
| ^
compilation terminated.
`gcc' failed in phase `C pre-processor'. (Exit code: 1)
)Can you plaese run this command from ghc directory with sudo: inplace/bin/ghc-stage1 -M -hisuf hi -osuf o -hcsuf hc -static -hide-all-packages -no-user-package-db '-this-unit-id integer-gmp-1.0.1.0' '-package-id ghc-prim-0.5.2.0' -i -i_build/stage1/libraries/integer-gmp -i_build/stage1/libraries/integer-gmp/build/autogen -ilibraries/integer-gmp/src/ -Iincludes -I_build/generated -I_build/stage1/libraries/integer-gmp -Ilibraries/integer-gmp/include -Ilibraries/integer-gmp/ -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/build/ -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -I_build/generated -optc-I_build/generated -optP-include -optP_build/stage1/libraries/integer-gmp/build/autogen/cabal_macros.h -optc-fno-stack-protector -odir _build/stage1/libraries/integer-gmp -hidir _build/stage1/libraries/integer-gmp -stubdir _build/stage1/libraries/integer-gmp -include-pkg-deps -dep-makefile build/stage1/libraries/integer-gmp/.dependencies.mk -dep-suffix '' -dep-suffix p -dep-suffix dyn_ libraries/integer-gmp/src/GHC/Integer.hs libraries/integer-gmp/src/GHC/Integer/GMP/Internals.hs libraries/integer-gmp/src/GHC/Integer/Logarithms.hs libraries/integer-gmp/src/GHC/Integer/Logarithms/Internals.hs libraries/integer-gmp/src/GHC/Integer/Type.hs -O2 -H32m -this-unit-id integer-gmp -Wall -XHaskell2010 -Wno-deprecated-flags
izgzhen
commented
6 years ago @chitrak7 same result:
libraries/integer-gmp/src/GHC/Integer/Type.hs:28:0: error:
fatal error: HsIntegerGmp.h: No such file or directory
#include "HsIntegerGmp.h"
^
|
28 | #include "HsIntegerGmp.h"
| ^
compilation terminated.
`gcc' failed in phase `C pre-processor'. (Exit code: 1)-> % find . -name "HsIntegerGmp.h"
./_build/stage1/libraries/integer-gmp/build/include/HsIntegerGmp.hI think we should include this as well...Err, bug is everywhere
chitrak7
commented
6 years ago -Ilibraries/integer-gmp//home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/build/include Its already included, but wrongly.
chitrak7
commented
6 years ago @izgzhen odd, I replaced /zhen/repos to /chitrak in your command and ran it on my terminal. It worked. Are you sure you executed it from ghc and not hadrian?
izgzhen
commented
6 years ago Are you sure you executed it from ghc and not hadrian
sure, otherwise the inplace/bin/ghc-stage1 won't be found.
izgzhen
commented
6 years ago @chitrak7 https://github.com/snowleopard/hadrian/blob/master/src/Settings/Builders/Common.hs#L33 this line should be the one producing the wrong -I as you mentioned above.
chitrak7
commented
6 years ago @izgzhen
Yeah! silly me :p. But I still believe the error is coming because of that inclusion only. All we need to do is add an additional path "_build/stage1/libraries/integer-gmp/build/include" to resolve this issue. I belive there is some typographical mistake here only. Can you retry by including this path as well, and removing the path "-Ilibraries/integer-gmp//home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/build/include".
chitrak7
commented
6 years ago @izgzhen The issue is much deeper. Actually package-data.mk is supposed to keep these paths in the category INCLUDE_DEPS. But unfortunately, there one is relative and other absolute. Please refer to my earlier comment: 2) A discrepancy in INCLUDE_DIRS in package-data.mk The file package-data.mk is supposed to give relative paths in category "INCLUDE_DIRS" while compiling a c file. This is evident from the code:
pure [ "-I" ++ pkgPath pkg -/- dir | dir <- incDirs ] But here is how directories are listed in the file:
include /home/chitrak/ghc/_build/stage1/libraries/base/build/include Here, "include" is relative while "/home/chitrak/ghc/_build/stage1/libraries/base/build/include" is absolute.
izgzhen
commented
6 years ago Oh, I see why you patch looks like that. I am waiting for my build to finish with your patch. Thanks!
chitrak7
commented
6 years ago Hopefully my patch will give error with process or unix library. Do inform me about the result. Also, did the HsIntegerConfig.hs error resolve by updating commands?
izgzhen
commented
6 years ago @chitrak7 My build stopped at ftruncate or _chsize error. HsIntegerConfig is fine. Finally synced haha.
chitrak7
commented
6 years ago With process library?? Or unix library? Can you give the final command.
izgzhen
commented
6 years ago process library. I am trying out the change you described above now, wait a bit.
izgzhen
commented
6 years ago @chitrak7
old: /usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/process -I_build/stage1/libraries/process/build -Ilibraries/process/include -Ilibraries/process//home/zhen/repos/ghc/_build/stage1/libraries/process/build/include -Iinclude -I/home/zhen/repos/ghc/_build/stage1/libraries/process/build/include -I/home/zhen/repos/ghc/libraries/directory -I/home/zhen/repos/ghc/_build/stage1/libraries/directory -I/home/zhen/repos/ghc/libraries/unix/include -I/home/zhen/repos/ghc/libraries/unix/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/unix/include -I/home/zhen/repos/ghc/_build/stage1/libraries/unix/dist-install/build/include -I/home/zhen/repos/ghc/libraries/time/lib/include -I/home/zhen/repos/ghc/libraries/time/lib/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/time/lib/include -I/home/zhen/repos/ghc/_build/stage1/libraries/time/lib/dist-install/build/include -I/home/zhen/repos/ghc/libraries/bytestring/include -I/home/zhen/repos/ghc/libraries/bytestring/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/bytestring/include -I/home/zhen/repos/ghc/_build/stage1/libraries/bytestring/dist-install/build/include -I/home/zhen/repos/ghc/libraries/base/include -I/home/zhen/repos/ghc/libraries/base/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/base/include -I/home/zhen/repos/ghc/_build/stage1/libraries/base/dist-install/build/include -I/home/zhen/repos/ghc/libraries/integer-gmp/include -I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/process/c/cbits/runProcess.p_o.d -MT _build/stage1/libraries/process/c/cbits/runProcess.o -x c libraries/process/cbits/runProcess.c
=====
new: /usr/bin/gcc -fno-stack-protector -Iincludes -I_build/generated -I_build/stage1/libraries/process -I_build/stage1/libraries/process/build -Ilibraries/process/include -Ilibraries/process//home/zhen/repos/ghc/_build/stage1/libraries/process/build/include -Iinclude -I/home/zhen/repos/ghc/_build/stage1/libraries/process/build/include -I/home/zhen/repos/ghc/libraries/directory -I/home/zhen/repos/ghc/_build/stage1/libraries/directory -I/home/zhen/repos/ghc/libraries/unix/include -I/home/zhen/repos/ghc/libraries/unix/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/unix/include -I/home/zhen/repos/ghc/_build/stage1/libraries/unix/dist-install/build/include -I/home/zhen/repos/ghc/libraries/time/lib/include -I/home/zhen/repos/ghc/libraries/time/lib/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/time/lib/include -I/home/zhen/repos/ghc/_build/stage1/libraries/time/lib/dist-install/build/include -I/home/zhen/repos/ghc/libraries/bytestring/include -I/home/zhen/repos/ghc/libraries/bytestring/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/bytestring/include -I/home/zhen/repos/ghc/_build/stage1/libraries/bytestring/dist-install/build/include -I/home/zhen/repos/ghc/libraries/base/include -I/home/zhen/repos/ghc/libraries/base/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/base/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/base/dist-install/build/include -I/home/zhen/repos/ghc/libraries/integer-gmp/include -I/home/zhen/repos/ghc/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/include -I/home/zhen/repos/ghc/_build/stage1/libraries/integer-gmp/dist-install/build/include -I/home/zhen/repos/ghc/_build/stage1/rts -I/home/zhen/repos/ghc/includes -I/home/zhen/repos/ghc/_build/generated -E -MM -MG -MF _build/stage1/libraries/process/c/cbits/runProcess.p_o.d -MT _build/stage1/libraries/process/c/cbits/runProcess.o -x c libraries/process/cbits/runProcess.cthe diff I made is changing -I/home/zhen/repos/ghc/_build/stage1/libraries/base/include to -I/home/zhen/repos/ghc/_build/stage1/libraries/base/build/include
chitrak7
commented
6 years ago @izgzhen You will need to add -I/home/zhen/repos/ghc/_build/stage1/libraries/base/build too. It is required for HsBase.hs
izgzhen
commented
6 years ago it is true -- how can we generalize this solution?
chitrak7
commented
6 years ago @izgzhen Please take a look at package-data.mk file of any library. For example, with process it is at _build/stage1/linraries/process/package-data.mk. We have to rewrite package-data.mk. We have to 1) Remove descrepancy in INCLUDE_DIRS 2) Remove bad directories from DEP_INCLUDE_DIRS 3) Add required directories to DEP_INCLUDE_DIRS. Can you help me how to do this task?
izgzhen
commented
6 years ago I’m not sure I know the solution but I guess it is related to how package-data.mk is formulated by Hadrian. I’ll take further look once I got some time ;)
snowleopard
commented
6 years ago @izgzhen @chitrak7 Thank you for investigating this further -- I hope I'll soon find some time to join! Just wanted to quickly add that we have a function that post-processes package-data files that are generated by ghc-cabal:
https://github.com/snowleopard/hadrian/blob/master/src/Rules/PackageData.hs#L100-L117
Maybe this is what needs to be tweaked.
Note that this issue will likely be gone after #445 is merged, since there will no longer be any ghc-cabal!
angerman
commented
6 years ago Note that this issue will likely be gone after #445 is merged, since there will no longer be any ghc-cabal!
Yes. @alpmestan has a working variant of #445 that drops ghc-cabal only (read: a subset of #445). Maybe just try to focus on that, and rather improve that new PR instead of trying to beat on ghc-cabal?
./build.sh -j --verbose --no-progress --progress-colour=never --progress-info=brief --profile=-