Open erlandsona opened 1 year ago
@erlandsona, I tried to reproduce this with a simpler example, but without success. My example was a two-package project (packageA
, packageB
), package A with a library (module LibA
) and package B with an executable depending on LibA
. So, project stack.yaml
:
resolver: lts-21.3
packages:
- packageB
- packageA
and package B's package.yaml
:
name: packageB
version: 0.1.0.0
dependencies:
- base >= 4.7 && < 5
- packageA
executables:
packageB:
main: Main.hs
source-dirs: app
If I change a file in package A and build the package B component (extracts only):
> stack build :packageB
packageA-0.1.0.0: unregistering (local file changes: src\LibA.hs)
packageA> build (lib)
...
packageA> Registering library for packageA-0.1.0.0..
packageB> build (exe)
Preprocessing executable 'packageB' for packageB-0.1.0.0..
Building executable 'packageB' for packageB-0.1.0.0..
[1 of 2] Compiling Main [LibA changed]
[3 of 3] Linking .stack-work\dist\c3556505\build\packageB\packageB.exe [Objects changed]
packageB> copy/register
Installing executable packageB in C:\Users\mikep\Documents\Code\Haskell\multiTest\.stack-work\install\4ce79b7a\bin
Completed 2 action(s).
So, in my example, the change to file src\LibA.hs
is being recognised.
Can we narrow down what you are experiencing? What file in your common
(or dp
) package are you changing that is not being detected when you build the :web
component?
We'll so I can show the log output without debug and maybe it'll be clearer. Issue isn't that stack doesn't show the change as detected and then hypothetically it recompiles packageB. It's that at runtime the previous version of packageA is still used.
Not at my machine rn, but I can try to create a small repro tomorrow.
Maybe try updating the string of a log statement in packageA then run packageB then update packageA again and see if the updates take effect.
@erlandsona, your reference to behaviour at run time has confused me, as that behaviour will depend on what source code has been compiled into the running executable file. To more fully specify my simple example: package A's LibA.hs
is:
module LibA where
someFuncA :: IO ()
someFuncA = putStrLn "Message: A-1"
and package B's executable is:
module Main where
import LibA
main :: IO ()
main = someFuncA
If I change the message in the source code of someFuncA
but do not then recompile the executable using that changed source code, of course, at run time there is no change in the executable's behaviour.
Issue isn't that we're not recompiling the executable but that stack install :the_project_executable
isn't.
Okay! Got a working reproduction of the issue!
When the log statement in common in changed the executable that I expect to have the latest log statement is still using a stale log.
Between the "REPRO" commit and the one prior, the primary thing I added was a "3rd project". EG: I wasn't able to repro the behavior with just 2 projects. I needed that 3rd "common" library with it's own Lib and I'm referencing it as a dependency in both projectA & projectB but when recompiling projectB, altho it seems to pick up the change the binary has stale code.
Okay new info from some more tinkering. I can get rid of the "issue" by removing the fan out. In other words, (using <- to mean "depends on") if projectB <- projectA <- common, then all is fine. BUT if projectB <- common && projectB <- projectA <- common...
There's 2 paths to get to common and when common is changed but projectA is not, and because there's no direct reference to common in projectB for that functionality, I think stack is picking the longer path to common through projectA which hasn't changed therefor we end up with the stale reference.
@erlandsona, I've cloned your example repository, but I still can't reproduce the problem. When I change common's Lib.hs
to have a different message (Common 4
), stack build :packageB
detects the changed file and everything behaves as it should (extracts below):
> stack build :projectB # Build only project B's executable
common-0.1.0.0: unregistering (local file changes: src\Lib.hs)
projectA-0.1.0.0: unregistering (missing dependencies: common)
projectB-0.1.0.0: unregistering (missing dependencies: common, projectA)
common > build (lib)
...
common > Registering library for common-0.1.0.0..
projectA> build (lib)
...
projectB> build (lib + exe)
...
Building executable 'projectB' for projectB-0.1.0.0..
...
Installing executable projectB in C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\bin
Registering library for projectB-0.1.0.0..
Completed 3 action(s).
> stack exec -- projectB # Run projectB executable
Common 4
ProjectA 1
EDIT: What output are you experiencing, when you do something similar?
So coming back to this today... what's weird is I had it reproduced without the --fast
flag but I came back to it this morning and all of a sudden the only way to trigger the weird caching behavior is to use --fast
then after the caches go stale the flag doesn't matter.
Couple other variables include:
In the following output, I expect "Common 2" to change to "Common 3" but notice in the 2nd output it still prints "Common 2".
$ stack install :projectB --fast --local-bin-path ./bin; bin/projectB
common-0.1.0.0: unregistering (local file changes: src/Lib.hs)
projectA-0.1.0.0: unregistering (missing dependencies: common)
projectB-0.1.0.0: unregistering (missing dependencies: common, projectA)
common > configure (lib)
common > Configuring common-0.1.0.0...
common > build (lib)
common > Preprocessing library for common-0.1.0.0..
common > Building library for common-0.1.0.0..
common > [1 of 2] Compiling Lib [Source file changed]
common > [2 of 2] Compiling Paths_common [Optimisation flags changed]
common > copy/register
common > Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY
common > Registering library for common-0.1.0.0..
projectA> configure (lib)
projectA> Configuring projectA-0.1.0.0...
projectA> build (lib)
projectA> Preprocessing library for projectA-0.1.0.0..
projectA> Building library for projectA-0.1.0.0..
projectA> [1 of 2] Compiling Lib [Optimisation flags changed]
projectA> [2 of 2] Compiling Paths_projectA [Optimisation flags changed]
projectA> copy/register
projectA> Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN
projectA> Registering library for projectA-0.1.0.0..
projectB> configure (lib + exe)
Configuring projectB-0.1.0.0...
projectB> build (lib + exe)
Preprocessing library for projectB-0.1.0.0..
Building library for projectB-0.1.0.0..
[1 of 2] Compiling Lib [Optimisation flags changed]
[2 of 2] Compiling Paths_projectB [Optimisation flags changed]
Preprocessing executable 'projectB' for projectB-0.1.0.0..
Building executable 'projectB' for projectB-0.1.0.0..
[1 of 2] Compiling Main [Optimisation flags changed]
[2 of 2] Compiling Paths_projectB [Optimisation flags changed]
[3 of 3] Linking .stack-work/dist/x86_64-linux/Cabal-3.8.1.0/build/projectB/projectB [Objects changed]
projectB> copy/register
Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr
Installing executable projectB in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/bin
Registering library for projectB-0.1.0.0..
Completed 3 action(s).
Copying from /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/bin/projectB to
/home/erlandsona/code/jenga/bin/projectB.
Copied executables to /home/erlandsona/code/jenga/bin/:
* projectB
Warning: Installation path /home/erlandsona/code/jenga/bin not found on the PATH environment variable.
Common 2
ProjectA 1
Bump to "Common 3"
$ stack install :projectB --fast --local-bin-path ./bin; bin/projectB
common-0.1.0.0: unregistering (local file changes: src/Lib.hs)
projectA-0.1.0.0: unregistering (missing dependencies: common)
projectB-0.1.0.0: unregistering (missing dependencies: common, projectA)
common > build (lib)
common > Preprocessing library for common-0.1.0.0..
common > Building library for common-0.1.0.0..
common > [1 of 2] Compiling Lib [Source file changed]
common > copy/register
common > Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY
common > Registering library for common-0.1.0.0..
projectA> build (lib)
projectA> Preprocessing library for projectA-0.1.0.0..
projectA> Building library for projectA-0.1.0.0..
projectA> copy/register
projectA> Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN
projectA> Registering library for projectA-0.1.0.0..
projectB> build (lib + exe)
Preprocessing library for projectB-0.1.0.0..
Building library for projectB-0.1.0.0..
Preprocessing executable 'projectB' for projectB-0.1.0.0..
Building executable 'projectB' for projectB-0.1.0.0..
projectB> copy/register
Installing library in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/lib/x86_64-linux-ghc-9.4.5/projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr
Installing executable projectB in /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/bin
Registering library for projectB-0.1.0.0..
Completed 3 action(s).
Copying from /home/erlandsona/code/jenga/.stack-work/install/x86_64-linux/98b826d8eec4182f6e19a539ba0f2c6b4bff041ba7e189ad84465ac6f69fe0f5/9.4.5/bin/projectB to
/home/erlandsona/code/jenga/bin/projectB.
Copied executables to /home/erlandsona/code/jenga/bin/:
* projectB
Warning: Installation path /home/erlandsona/code/jenga/bin not found on the PATH environment variable.
Common 2
ProjectA 1
@erlandsona, that is indeed very odd (by the way, I've been testing on Windows; I'll switch to Ubuntu). When you bump the source, everything seems to get recompiled - as I would expect - but the final bin/projectB
seems to be - somehow - still running the 'old' executable.
Can we take the --copy-bins
(stack install
) step out of the equation and just try stack build
followed by stack exec -- projectB
? I want to be sure that the executable file that is being run is, indeed, the executable file that Stack just built.
The --fast
flag literally just adds -O0
to the GHC options (Stack.Options.BuildParser.buildOptsParser
) - I find it hard to fathom how it could be involved.
@erlandsona, good news, I can reproduce what you are experiencing on Windows and Ubuntu 22.04.1 LTS (WSL 2), and it does seem to be somehow connected with the --fast
flag.
Phew!!! Glad I'm not sending yah on a wild goose chase!
Yeh, something to do with the --fast
flag, but on our larger project with this sort of setup we're able to reproduce it without the flag as well. Of course, after a stack clean
everything builds fine but the compile times are an order of magnitude off such that it's prohibitively expensive to stack clean
before every re-compile when we're iterating on Lucid HTML templates for example.
I also tried you're suggestion with stack build :projectB --fast; stack exec -- projectB
and was able to repro the issue.
I've compared the corresponding output (outputf1.txt
versus output1.txt
; and outputf2.txt
versus output2.txt
) of:
> stack purge
> stack --verbose --no-time-in-log install :projectB --fast --local-bin-path ./bin 2>&1 > outputf1.txt ; bin/projectB
> # Bump message in common's Lib.hs
> stack --verbose --no-time-in-log install :projectB --fast --local-bin-path ./bin 2>&1 > outputf2.txt ; bin/projectB
and
> stack purge
> stack --verbose --no-time-in-log install :projectB --local-bin-path ./bin 2>&1 > output1.txt ; bin/projectB
> # Bump message in common's Lib.hs
> stack --verbose --no-time-in-log install :projectB --local-bin-path ./bin 2>&1 > output2.txt ; bin/projectB
outputf1.txt
has the --ghc-options -O0
in the Cabal configure
steps which the output1.txt
does not (as expected).
outputf2.txt
seems to me to have different behaviour of Cabal (the library) compared to output2.txt
, despite Stack passing the same commands to Cabal in each case. I am going to see what --cabal-verbose
yields.
Not sure what stack purge
does but if it does a stack clean
under the hood then your test might be flaky given clean
"fixes" the bug. So you're not actually outputting a diff between the two.
Another test:
> stack purge # Start with a clean project
> stack build common:lib
> stack exec -- ghc-pkg field common abi
abi: 05c3e02960afdc567cf6780e0c4efa27
> # Bump message in common's Lib.hs
> stack build common:lib
> stack exec -- ghc-pkg field common abi # ABI hash of the registered package changes, as expected ...
abi: 85127eacbbc7934532c98093a375e836
but ...
> stack purge # Start with a clean project
> stack build common:lib --fast
> stack exec -- ghc-pkg field common abi
abi: 71b6f976c20ac17f186c8979b443038b
> # Bump message in common's Lib.hs
> stack build common:lib --fast
> stack exec -- ghc-pkg field common abi # !!! the ABI hash of the registered package is unchanged !!!
abi: 71b6f976c20ac17f186c8979b443038b
Something seems to be going wrong with the registration step when --fast
is specified ... (EDIT: It is not --fast
specific; the same behaviour results if --ghc-options -O0
is used expressly.)
My current hypothesis is that this arises because - with -O0
- changes to the source code do not affect the *.hi
file or the ABI hash for the package. With optimisation greater than O0
, parts of the source code are included in the *.hi
file and the ABI hash for the package changes as the source code changes.
I have been investigating with the simplest of packages: library-only, single module Lib
:
module Lib
( someFunc
) where
someFunc :: IO ()
someFunc = putStrLn "someFunc1"
and inspecting the resulting Lib.hi
file with stack exec -- ghc --show-iface C:\Users\mikep\Documents\Code\Haskell\abiTest\.stack-work\dist\c3556505\build\Lib.hi
as "someFunc1"
is changed to "someFunc2"
.
With -O0
, the src_hash
changes but the ABI hash is constant. With some optimisation, Lib.hi
includes some of the source code (for example, extract below) and the src_hash
and ABI hash changes:
b677cd349fe756bad7775c5130ce9be8
someFunc3 :: GHC.Prim.Addr#
[HasNoCafRefs, LambdaFormInfo: LFUnlifted,
Unfolding: ("someFunc1"#)]
More experimentation, with the jenga
example (but after renaming the A and B 'Lib' modules to give them distinct names) (extracts only, some reformatting). It seems that it is GHC 9.4.5 that concludes that it does not need to 'recompile' and it can 'skip' because the 'fingerprint' is unchanged):
> # Turn on Cabal and GHC verbosity...
> stack build --ghc-options "-O0 -v -ddump-hi-diffs" --cabal-verbose --local-bin-path ./bin; bin/projectB
common-0.1.0.0: unregistering (local file changes: src\Lib.hs)
projectA-0.1.0.0: unregistering (missing dependencies: common)
projectB-0.1.0.0: unregistering (missing dependencies: common, projectA)
common > build (lib)
...
common > Installing .stack-work\dist\c3556505\build\Lib.hi to
common > C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\lib\x86_64-windows-ghc-9.4.5\common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY\Lib.hi
...
common > Installing
common > .stack-work\dist\c3556505\build\libHScommon-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY.a
common > to
common > C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\lib\x86_64-windows-ghc-9.4.5\common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY\libHScommon-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY.a
...
projectA> build (lib)
...
projectA> "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.5\bin\ghc-9.4.5.exe"
"--make"
"-fbuilding-cabal-package"
"-O"
"-outputdir" ".stack-work\dist\c3556505\build"
"-odir" ".stack-work\dist\c3556505\build"
"-hidir" ".stack-work\dist\c3556505\build"
"-stubdir" ".stack-work\dist\c3556505\build"
"-i"
"-i.stack-work\dist\c3556505\build"
"-isrc"
"-i.stack-work\dist\c3556505\build\autogen"
"-i.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\autogen"
"-I.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build"
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include" "-optP.stack-work\dist\c3556505\build\autogen\cabal_macros.h"
"-this-unit-id" "projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN"
"-hide-all-packages"
"-Wmissing-home-modules"
"-no-user-package-db"
"-package-db" "C:\sr\snapshots\ce334f6e\pkgdb"
"-package-db" "C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\pkgdb"
"-package-db" ".stack-work\dist\c3556505\package.conf.inplace"
"-package-id" "base-4.17.1.0"
"-package-id" "common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY"
"-XHaskell2010"
"LibA"
"-O0"
"-v"
"-ddump-hi-diffs"
"-fhide-source-paths"
"-fdiagnostics-color=always"
projectA> Glasgow Haskell Compiler, Version 9.4.5, stage 2 booted by GHC version 9.2.2
...
projectA> *** Checking old interface for LibA (use -ddump-hi-diffs for more details):
projectA> Considering whether compilation is required for LibA:
projectA> Module flags unchanged
projectA> Optimisation flags unchanged
projectA> HPC flags unchanged
projectA> signatures to merge in unchanged
projectA> []
projectA> implementing module unchanged
projectA> Checking interface for module Prelude base
projectA> Module fingerprint unchanged
projectA> Checking interface for module System.IO base
projectA> Module fingerprint unchanged
projectA> Checking interface for module Lib common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY
projectA> Module fingerprint unchanged
projectA> Checking interface for module GHC.Types ghc-prim
projectA> Module fingerprint unchanged
projectA> [1 of 1] Skipping LibA
...
projectB> build (lib + exe)
...
projectB> "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.5\bin\ghc-9.4.5.exe"
"--make"
"-fbuilding-cabal-package"
"-O"
"-outputdir" ".stack-work\dist\c3556505\build"
"-odir" ".stack-work\dist\c3556505\build"
"-hidir" ".stack-work\dist\c3556505\build"
"-stubdir" ".stack-work\dist\c3556505\build"
"-i"
"-i.stack-work\dist\c3556505\build"
"-isrc"
"-i.stack-work\dist\c3556505\build\autogen"
"-i.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\autogen"
"-I.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build"
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include" "-optP.stack-work\dist\c3556505\build\autogen\cabal_macros.h"
"-this-unit-id" "projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr"
"-hide-all-packages"
"-Wmissing-home-modules"
"-no-user-package-db"
"-package-db" "C:\sr\snapshots\ce334f6e\pkgdb"
"-package-db" "C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\pkgdb"
"-package-db" ".stack-work\dist\c3556505\package.conf.inplace"
"-package-id" "base-4.17.1.0"
"-package-id" "common-0.1.0.0-JWbNpAqBj0sJ4NuqRU3aZY"
"-package-id" "projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN"
"-XHaskell2010"
"LibB"
"-O0"
"-v"
"-ddump-hi-diffs"
"-fhide-source-paths"
"-fdiagnostics-color=always"
projectB> Glasgow Haskell Compiler, Version 9.4.5, stage 2 booted by GHC version 9.2.2
...
projectB> *** Checking old interface for LibB (use -ddump-hi-diffs for more details):
projectB> Considering whether compilation is required for LibB:
projectB> Module flags unchanged
projectB> Optimisation flags unchanged
projectB> HPC flags unchanged
projectB> signatures to merge in unchanged
projectB> []
projectB> implementing module unchanged
projectB> Checking interface for module Prelude base
projectB> Module fingerprint unchanged
projectB> Checking interface for module LibA projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN
projectB> Module fingerprint unchanged
projectB> [1 of 1] Skipping LibB
...
projectB> "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.5\bin\ghc-9.4.5.exe"
"--make"
"-no-link"
"-fbuilding-cabal-package"
"-O"
"-static"
"-outputdir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-odir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-hidir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-stubdir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-i"
"-i.stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-iapp"
"-i.stack-work\dist\c3556505\build\projectB\autogen"
"-i.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\projectB\autogen"
"-I.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include" "-optP.stack-work\dist\c3556505\build\projectB\autogen\cabal_macros.h"
"-hide-all-packages"
"-Wmissing-home-modules"
"-no-user-package-db"
"-package-db" "C:\sr\snapshots\ce334f6e\pkgdb"
"-package-db" "C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\pkgdb"
"-package-db" ".stack-work\dist\c3556505\package.conf.inplace"
"-package-id" "base-4.17.1.0"
"-package-id" "projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr"
"-XHaskell2010"
"app\Main.hs"
"-threaded"
"-rtsopts"
"-with-rtsopts=-N"
"-O0"
"-v"
"-ddump-hi-diffs"
"-fhide-source-paths"
"-fdiagnostics-color=always"
projectB> Glasgow Haskell Compiler, Version 9.4.5, stage 2 booted by GHC version 9.2.2
...
projectB> *** Checking old interface for Main (use -ddump-hi-diffs for more details):
projectB> Considering whether compilation is required for Main:
projectB> Module flags unchanged
projectB> Optimisation flags unchanged
projectB> HPC flags unchanged
projectB> signatures to merge in unchanged
projectB> []
projectB> implementing module unchanged
projectB> Checking interface for module Prelude base
projectB> Module fingerprint unchanged
projectB> Checking interface for module GHC.Types ghc-prim
projectB> Module fingerprint unchanged
projectB> Checking interface for module LibA projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN
projectB> Module fingerprint unchanged
projectB> Checking interface for module LibB projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr
projectB> Module fingerprint unchanged
projectB> [1 of 1] Skipping Main
...
projectB> Linking...
projectB> "C:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\ghc-9.4.5\bin\ghc-9.4.5.exe"
"--make"
"-fbuilding-cabal-package"
"-O"
"-static"
"-outputdir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-odir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-hidir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-stubdir" ".stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-i"
"-i.stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-iapp"
"-i.stack-work\dist\c3556505\build\projectB\autogen"
"-i.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\projectB\autogen"
"-I.stack-work\dist\c3556505\build\global-autogen"
"-I.stack-work\dist\c3556505\build\projectB\projectB-tmp"
"-IC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\include"
"-optP-include"
"-optP.stack-work\dist\c3556505\build\projectB\autogen\cabal_macros.h"
"-LC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\lib"
"-LC:\Users\mikep\AppData\Local\Programs\stack\x86_64-windows\msys2-20230526\mingw64\bin"
"-hide-all-packages"
"-Wmissing-home-modules"
"-no-user-package-db"
"-package-db" "C:\sr\snapshots\ce334f6e\pkgdb"
"-package-db" "C:\Users\mikep\Documents\Code\Haskell\jenga\.stack-work\install\4ce79b7a\pkgdb"
"-package-db" ".stack-work\dist\c3556505\package.conf.inplace"
"-package-id" "base-4.17.1.0"
"-package-id" "projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr"
"-XHaskell2010"
"app\Main.hs"
"-o" ".stack-work\dist\c3556505\build\projectB\projectB.exe"
"-threaded"
"-rtsopts"
"-with-rtsopts=-N"
"-O0"
"-v"
"-ddump-hi-diffs"
"-fhide-source-paths"
"-fdiagnostics-color=always"
projectB> Glasgow Haskell Compiler, Version 9.4.5, stage 2 booted by GHC version 9.2.2
...
projectB> *** Checking old interface for Main (use -ddump-hi-diffs for more details):
projectB> Considering whether compilation is required for Main:
projectB> Module flags unchanged
projectB> Optimisation flags unchanged
projectB> HPC flags unchanged
projectB> signatures to merge in unchanged
projectB> []
projectB> implementing module unchanged
projectB> Checking interface for module Prelude base
projectB> Module fingerprint unchanged
projectB> Checking interface for module GHC.Types ghc-prim
projectB> Module fingerprint unchanged
projectB> Checking interface for module LibA projectA-0.1.0.0-3vHJZZMyKjL74RQQ7ayBN
projectB> Module fingerprint unchanged
projectB> Checking interface for module LibB projectB-0.1.0.0-IAxteA2fuQBCVNZMliTzVr
projectB> Module fingerprint unchanged
projectB> [1 of 2] Skipping Main
...
projectB> link: linkables are ...
projectB> LinkableM (2023-07-22 15:16:10.3480018 UTC) Main
projectB> [DotO .stack-work\dist\c3556505\build\projectB\projectB-tmp\Main.o]
projectB> .stack-work\dist\c3556505\build\projectB\projectB.exe is up to date, linking not required.
projectB> [2 of 2] Skipping .stack-work\dist\c3556505\build\projectB\projectB.exe
...
Completed 3 action(s).
Common 1
ProjectA 1
@erlandsona, I am now fairly certain that this is a feature/bug of GHC 9.4.5. It seems that GHC's 'recompilation checker' assumes that recompilation of a module is not required if the ABI hash of its imports has not changed, even if the import module's source code has changed - and that is more likely to happen with --fast
/-O0
. GHC's recompilation checker can be turned off with GHC flag -fforce-recomp
. Consequently, I have (extracts only):
> stack purge # Start with a project purged of build artefacts
> stack install :projectB --fast --ghc-options -fforce-recomp --local-bin-path ./bin; bin/projectB
...
Common 1
ProjectA 1
> # Bump to "Common 2" and experience the problem with --fast/-O0
> stack install :projectB --fast --local-bin-path ./bin; bin/projectB
...
Common 1
ProjectA 1
> # Bump to "Common 3" and force GHC to recompile
> stack install :projectB --fast --ghc-options -fforce-recomp --local-bin-path ./bin; bin/projectB
...
Common 3
ProjectA 1
I'll follow up elsewhere to see if I can determine if this is a feature or a bug of GHC.
I came across https://github.com/haskell/cabal/issues/4381 and found it helpful to understand Cabal's use of 'installed package ID', which I could not find documentation for outside of https://cabal.readthedocs.io/en/stable/developing-packages.html#package-names-and-versions and "There is a separate installed package ID that uniquely identifies each installed package instance. Most of the time however, users need not be aware of this detail."
@mpilgrem we're definitely keeping a close eye on the progress of that issue you mentioned and we'll give the --ghc-options -fforce-recomp
a shot and see how that effects compile times :grimacing:
Thanks for all your help with this!
So we tried the --ghc-options -fforce-recomp
and it balloons incremental recompilation from 7sec
-> 33sec
.
Which makes sense but is unfortunately not sufficient for our needs (authoring HTML view code).
Our current solution is to just put all the modules under a single package.yaml / library such that this bug doesn't surface.
Of course this has the drawback that any external repo's depending on this project have to compile unnecessary modules in order to leverage "shared types".
@erlandsona, there is now a related discussion at the Haskell Community. It appears that this may be a GHC bug, introduced in GHC 9.4.5 (or some other GHC 9.4 version). If you can use GHC 9.2.8, I think it may be free of the bug.
There's also now a ghc bug: https://gitlab.haskell.org/ghc/ghc/-/issues/23724
Thanks @mpilgrem and @erlandsona for figuring this out. We just upgraded to GHC 9.4.5, and I thought I was losing my mind for a bit there 😂
Stack doesn't seem to be properly cache busting when cross project dependencies change.
Steps to reproduce
Example: $project_root/stack.yaml
$project_root/common/package.yaml
$project_root/db/package.yaml
$project_root/web/package.yaml
Expected
When a change is made to common or db those changes buste the cache of web such that web is recompiled with the latest changes from it's dependencies.
Actual
web seems to be stuck recompiling with stale cached data such that changes in common / db aren't propagating properly.
If you suspect that a Stack command misbehaved, please include the output of that command in
--verbose
mode. If the output is larger than a page please paste the output in a Gist.Stack version
Method of installation
Platform
Your platform (machine architecture and operating system)