commercialhaskell / stack

The Haskell Tool Stack
http://haskellstack.org
BSD 3-Clause "New" or "Revised" License
3.95k stars 842 forks source link

Why does the stack script setup take a long time? #6588

Closed philderbeast closed 1 month ago

philderbeast commented 1 month ago

I'm updating the GitHub actions for hpack-dhall and noticed a slowness related to Stack and its scripts.

    runs-on: ubuntu-latest
    strategy:
      matrix:
        cabal: ["3.10.3.0"]
        vers:
          - {ghc: "9.8.2", stackage: "nightly-2024-05-18"}

    - uses: haskell-actions/setup@v2
      name: setup GHC and cabal-install
      id: setup
      with:
        ghc-version: ${{ matrix.vers.ghc }}
        cabal-version: ${{ matrix.cabal }}
        enable-stack: true
        stack-version: '2.15.7'

Even though I have stack-2.15.7 installed, here I'm running Updo in script mode, using Stack's scripts.

    - name: Updo
      run: GHC_VERSION=${{ matrix.vers.ghc }} STACKAGE_VERSION=${{ matrix.vers.stackage }} \
        make -f project-files.mk cabal.project

Why does the following use Cabal-3.2.1.0 and ghc-8.10.7 for scripts? The linking seems to be taking a long time, nearly 10 mins and longer than the time it takes to install Dhall from source. I can likely avoid this with Updo by switching to installing its executables and using those instead of its scripts.

[1 of 2] Compiling Main             ( /home/runner/.stack/setup-exe-src/setup-HwdwpEmb.hs, /home/runner/.stack/setup-exe-src/setup-HwdwpEmb.o )
[2 of 2] Compiling StackSetupShim   ( /home/runner/.stack/setup-exe-src/setup-shim-HwdwpEmb.hs, /home/runner/.stack/setup-exe-src/setup-shim-HwdwpEmb.o )
Linking /home/runner/.stack/setup-exe-cache/x86_64-linux-tinfo6/tmp-Cabal-simple_HwdwpEmb_3.2.1.0_ghc-8.10.7 ...
mpilgrem commented 1 month ago

If I understand correctly, you are using this Haskell script file: https://github.com/cabalism/updo/blob/main/project-dhall/pkgs-sorted.hs. That starts with:

#!/usr/bin/env stack
-- stack script --resolver=lts-18.27 --package=base --package=dhall --package=filepath --package=text

Stackage snapshot LTS 18.27 specifies GHC 8.10.7. GHC 8.10.7's Cabal boot library is Cabal-3.2.1.0.

So, Stack has to install GHC 8.10.7 (if it has not already done so). That will take some time.

philderbeast commented 1 month ago

Thanks @mpilgrem, how could I forget, I hard-wired that in!

For Updo, we'd switched from using Stack scripts to Cabal scripts as that frees us from having to specify a resolver. After encountering some caching issues with Cabal scripts we switched back.

philderbeast commented 1 month ago

From the Stack script interpreter docs I may be able to set the resolver for the script on the fly to avoid needing two GHC versions for a Github actions run, either by editing the script on the fly or by not using the shebang and making an explicit call to stack pkgs-sorted.hs.

I'll need to experiment to see if setting the resolver on the command line can override the resolver set in the script and if the other -- stack script settings hold. I wonder if I can leave the --resolver out of the script and have it pick this up from the environment (or override it from the environment), perhaps from the STACK_YAML environment variable (there doesn't seem to be a STACK_RESOLVER enviroment variable).

mpilgrem commented 1 month ago

@philderbeast, I think the combination of:

means that a snapshot specified in a Stack interpreter options comment will prevail over one specified at the command line. I've not tested, but I think you may be able to drop the specification in the options comment (but would then have to include the specification in the command line).

philderbeast commented 4 weeks ago

I can use sed to use the resolver of my choosing, $(STACKAGE_VERSION), in those scripts as part of Updo's bootstrapping. Something like:

UPDO_VERSION ?= 011be290d1d7ebcb2f8776565c9b50c7c843ba77
UPDO_URL := https://github.com/cabalism/updo/archive/${UPDO_VERSION}.tar.gz

updo/Makefile:
    rm -rf updo
    curl -sSL ${UPDO_URL} | tar -xz
    mv updo-* updo
    chmod +x $$(grep -RIl '^#!' updo)
    grep -RIl '^#!' updo \
    | xargs sed --in-place -E 's/--resolver=(nightly-[[:digit:]]{4}-[[:digit:]]{2}-[[:digit:]]{2}|lts-[[:digit:]]+\.[[:digit:]]+)/--resolver=$(STACKAGE_VERSION)/g'