Missing (or bad) header file: fs.h: in unlit

[izgzhen]

hadrian: Missing dependency on a foreign library:
* Missing (or bad) header file: fs.h
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.If the
library file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
If the header file does exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.

shakeArgsWith   0.000s    0%                           
Function shake  0.218s    2%                           
Database read   0.000s    0%                           
With database   0.000s    0%                           
Running rules   7.386s   97%  =========================
Total           7.604s  100%                           
Error when running Shake build system:
* _build/stage1/bin/unlit
* OracleQ (PackageDataFile (Context {stage = Stage1, package = Package {pkgLanguage = Haskell, pkgType = Program, pkgName = "unlit", pkgPath = "utils/unlit"}, way = v}))
* _build/stage1/utils/unlit/setup-config
* _build/stage0/bin/unlit
* OracleQ (PackageDataFile (Context {stage = Stage0, package = Package {pkgLanguage = Haskell, pkgType = Program, pkgName = "unlit", pkgPath = "utils/unlit"}, way = v}))
* _build/stage0/utils/unlit/setup-config
ExitFailure 1

---

{'ghc-git-hash': '5c9bc0771f426de2b99d6bb3a0f1d2eeaa91c710', 'hadrian-git-hash': '64da39cc9f955d7c1492ab39927a8e0849b23f00', 'gcc-version': '5.4.0', 'clean': True, 'duration': 13.175835371017456, 'exit-code': 1, 'stage0-ghc-version': '8.2.2', 'timestamp': 'Tue Apr 24 15:23:21 2018', 'platform': 'Linux-4.13.0-38-generic-x86_64-with-Ubuntu-16.04-xenial', 'flavour': 'quickest'}

[izgzhen]

$ cat ../utils/unlit/unlit.cabal
cabal-version: 2.1
Name: unlit
Version: 0.1
Copyright: XXX
License: BSD-3-Clause
Author: XXX
Maintainer: XXX
Synopsis: Literate program filter
Description: XXX
Category: Development
build-type: Simple

Executable unlit
    Default-Language: Haskell2010
    Main-Is: unlit.c
    C-Sources: unlit.c, fs.c
    Includes: fs.h

[izgzhen]

@Mistuke similar to https://github.com/snowleopard/hadrian/issues/554

[izgzhen]

fs.c can't be found either

[Mistuke]

If they can't be found then something went wrong with your configure step. The Hadrian CI is finding them and building fine so I think it's a local issue.

Check the output of configure

On Tue, Apr 24, 2018, 09:18 Zhen Zhang notifications@github.com wrote:

fs.c can't be found either

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/snowleopard/hadrian/issues/579#issuecomment-383845634, or mute the thread https://github.com/notifications/unsubscribe-auth/ABH3KRM7SJ2FrJdgQk8kgpet7FzVtOZdks5trt_lgaJpZM4ThM6V .

[izgzhen]

Thanks! I removed everything outside the tree and trying to rebuild now

[izgzhen]

Check the output of configure

https://gist.github.com/izgzhen/1bb69b0aa67bc1e367b82da3d6f6dade

Still the same after cleaning up.

$ ls utils/unlit 
fs.c  fs.h  ghc.mk  Makefile  README  unlit.c  unlit.cabal

[snowleopard]

@izgzhen I've just restarted Travis to check if it might fail with GHC HEAD.

[Mistuke]

The make based version doesn't use the cabal file it looks like. But the cabal file looks correct to me and the files are in the right place. Not sure what the problem here is. Especially if it also can't find the C file.

On Tue, Apr 24, 2018, 11:10 Andrey Mokhov notifications@github.com wrote:

@izgzhen https://github.com/izgzhen I've just restarted Travis to check if it might fail with GHC HEAD.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/snowleopard/hadrian/issues/579#issuecomment-383878613, or mute the thread https://github.com/notifications/unsubscribe-auth/ABH3KbILWMAfmwL9c68EPsPX_e_78a7dks5trvowgaJpZM4ThM6V .

[snowleopard]

Looks like no CI bots hit this issue.

[Mistuke]

Something odd though. Why is configure being run at all for unlit. It's build type is Simple and it has no configure script.

[izgzhen]

I found what is going on! if I use -c of build.sh (instead of manually ./configure and then run build.sh without -c), it will be fine.

I used to ./configure, so I wonder if this is an expected behaviour?

[Mistuke]

At which level are you running configure? It may be copying to the wrong place. So what's your cwd in relation to the configure script?

[izgzhen]

At which level are you running configure? It may be copying to the wrong place. So what's your cwd in relation to the configure script?

I run ./configure at the GHC top-level

[snowleopard]

I used to ./configure, so I wonder if this is an expected behaviour?

@izgzhen Definitely not expected. Running boot + configure at the GHC root should be equivalent to running Hadrian with -c.

P.S.: The only difference is that Hadrian will track (some) .in files and will reconfigure if they change.

[Mistuke]

My guess is that the ln destination for the copies need to be prepend with $srcdir/ i think that would fix it. But i can't test till later.

On Tue, Apr 24, 2018, 13:51 Andrey Mokhov notifications@github.com wrote:

I used to ./configure, so I wonder if this is an expected behaviour?

@izgzhen https://github.com/izgzhen Definitely not expected. Running boot + configure at the GHC root should be equivalent to running Hadrian with -c.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/snowleopard/hadrian/issues/579#issuecomment-383918586, or mute the thread https://github.com/notifications/unsubscribe-auth/ABH3KVMfIYaRH9sLlHYJ6pGoppFzK6ALks5trx_egaJpZM4ThM6V .

[Mistuke]

@izgzhen can you try in your GHC source tree changing these lines https://github.com/ghc/ghc/blob/master/configure.ac#L676

so they look like

ln -f -v $srcdir/utils/fs/fs.* $srcdir/utils/lndir/

so appending $srcdir/ to each. If this doesn't work could you attach the config.status file for the working and not working builds? Thanks. All my source trees are currently dirty so I can't easily try myself.

[izgzhen]

@Mistuke appending $srcdir does work:

diff --git a/configure.ac b/configure.ac
index a8b8681a36..cf614e64eb 100644
--- a/configure.ac
+++ b/configure.ac
@@ -673,8 +673,8 @@ dnl --------------------------------------------------------------
 dnl ** Copy the files from the "fs" utility into the right folders.
 dnl --------------------------------------------------------------
 AC_MSG_NOTICE([Creating links for in-tree file handling routines.])
-ln -f -v utils/fs/fs.* utils/lndir/
-ln -f -v utils/fs/fs.* utils/unlit/
+ln -f -v $srcdir/utils/fs/fs.* $srcdir/utils/lndir/
+ln -f -v $srcdir/utils/fs/fs.* $srcdir/utils/unlit/
 ln -f -v utils/fs/fs.* rts/
 ln -f -v utils/fs/fs.h libraries/base/include/
 ln -f -v utils/fs/fs.c libraries/base/cbits/

[Mistuke]

Ah great, I'll submit a patch later today

On Wed, Apr 25, 2018, 01:28 Zhen Zhang notifications@github.com wrote:

@Mistuke https://github.com/Mistuke appending $srcdir does work:

diff --git a/configure.ac b/configure.ac index a8b8681a36..cf614e64eb 100644--- a/configure.ac+++ b/configure.ac@@ -673,8 +673,8 @@ dnl -------------------------------------------------------------- dnl * Copy the files from the "fs" utility into the right folders. dnl -------------------------------------------------------------- AC_MSG_NOTICE([Creating links for in-tree file handling routines.])-ln -f -v utils/fs/fs. utils/lndir/-ln -f -v utils/fs/fs. utils/unlit/+ln -f -v $srcdir/utils/fs/fs. $srcdir/utils/lndir/+ln -f -v $srcdir/utils/fs/fs. $srcdir/utils/unlit/ ln -f -v utils/fs/fs. rts/ ln -f -v utils/fs/fs.h libraries/base/include/ ln -f -v utils/fs/fs.c libraries/base/cbits/

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/snowleopard/hadrian/issues/579#issuecomment-384122063, or mute the thread https://github.com/notifications/unsubscribe-auth/ABH3KVqA41wX2YYlawZhy96HAV4km_t9ks5tr8MOgaJpZM4ThM6V .

[izgzhen]

@Mistuke wait a bit .. it might be a false positive. I was toying with cross compilation, and it failed in a similar way.

EDIT: false false positive maybe ... well, build system is kinda difficult to deal with

[Mistuke]

@izgzhen Can you tell me what I need to do to reproduce this? I'm not entirely sure how Hadrian does the out of tree build.

[izgzhen]

@izgzhen Can you tell me what I need to do to reproduce this? I'm not entirely sure how Hadrian does the out of tree build.

In ghc, ./boot and ./configure, then hadrian/build.sh --flavour=quickest

[izgzhen]

@Mistuke how is this going on now?

[Mistuke]

I've tried to reproduce it a couple of times and couldn't. I also don't understand why there would be any difference at all between running ./configure manually and hadrian -c. Also cabal is able to build the unlit package after a configure just fine. So I don't think this is a problem with these files.

[snowleopard]

@izgzhen I also couldn't reproduce this. Are you sure this happens if you simply checkout a fresh GHC tree and build it with Hadrian?

[izgzhen]

I can't reproduce this now either :( Thanks for the information! I will close this issue.

[jfischoff]

FWIW I'm running into this as well. It happens with the old make build too.

[snowleopard]

@jfischoff As you can see from the above discussion, this appears to be a hard-to-reproduce issue. If it persists, I suggest you report it on https://gitlab.haskell.org/ghc/ghc/issues, since Hadrian development moved to the GHC tree.

[ggreif]

Try to re-configure, I have seen header files disappearing when cleaning, but they got recreated on configure. It always struck me as odd.

On 5/2/20, Andrey Mokhov notifications@github.com wrote:

@jfischoff As you can see from the above discussion, this appears to be a hard-to-reproduce issue. If it persists, I suggest you report it on https://gitlab.haskell.org/ghc/ghc/issues, since Hadrian development moved to the GHC tree.

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/snowleopard/hadrian/issues/579#issuecomment-622988743

[jfischoff]

I think I figured out what the issue is. I am using a shared folder with vagrant and virtualbox. I changed the configuration to bidirectional syncing based on the "virtualbox" type. This disables hard links which caused an issue in setup I didn't see initially.

[snowleopard]

@jfischoff Good to hear you found a fix!

@ggreif Thanks for the hint :)