search

purescript / spago

🍝 PureScript package manager and build tool
BSD 3-Clause "New" or "Revised" License
792 stars 132 forks source link

Support incremental builds #1180

Closed deemp closed 10 months ago

deemp commented 10 months ago

Superseded by https://github.com/purescript/purescript/issues/3995

Related to https://github.com/purescript/spago/issues/700

Problem

Hi! spago seems to ignore already compiled files.

Steps to reproduce

  1. Clone this repo at master or at a commit, e.g. bbe37b6cd497aa544bd0761fa7a56a5f5d002a87.

  2. Remove the output directory.

  3. Run spago build.

    • you may see logs like this: 
      
Reading Spago workspace configuration...

    ✅ Selecting 6 packages to build: docs-search-client-halogen docs-search-common docs-search-index spago spago-bin spago-core

    Downloading dependencies... Building... [ 11 of 1014] Compiling Record.Unsafe [ 10 of 1014] Compiling Spago.Command.Script [ 9 of 1014] Compiling Web.DOM.Internal.Types [ 3 of 1014] Compiling Spago.Generated.BuildInfo [ 7 of 1014] Compiling Type.Function [ 2 of 1014] Compiling Type.Row [ 1 of 1014] Compiling Unsafe.Coerce [ 6 of 1014] Compiling Web.Event.Internal.Types [ 4 of 1014] Compiling Type.Proxy [ 8 of 1014] Compiling Type.Equality [ 5 of 1014] Compiling Spago.Texts [ 12 of 1014] Compiling Partial ...

  4. Interrupt the build (Ctrl+C in my case) before it finishes.

  5. Check the output directory (part of the logs is omitted).

    tree output
output
|-- Control.Semigroupoid
|   |-- corefn.json
|   |-- docs.json
|   |-- externs.cbor
|   |-- index.js
|   `-- index.js.map
|-- Data.ArrayBuffer.Types
|   |-- corefn.json
|   |-- docs.json
|   |-- externs.cbor
|   |-- index.js
|   `-- index.js.map
|-- Data.Boolean
|   |-- corefn.json
|   |-- docs.json
|   |-- externs.cbor
|   |-- index.js
|   `-- index.js.map

...

23 directories, 116 files

  6. Record the statistics on output/Control.Semigroupoid/index.js.

    $ stat output/Control.Semigroupoid/index.js
Access: 2024-01-25 01:19:46.011057433 +0300
Modify: 2024-01-25 01:32:12.667933173 +0300
Change: 2024-01-25 01:32:12.667933173 +0300
Birth: 2024-01-25 01:19:46.011057433 +0300

  7. Run spago build again.

    Reading Spago workspace configuration...

✅ Selecting 6 packages to build:
    docs-search-client-halogen
    docs-search-common
    docs-search-index
    spago
    spago-bin
    spago-core

Downloading dependencies...
Building...
[   3 of 1014] Compiling Web.DOM.Internal.Types
[   8 of 1014] Compiling Type.Function
[   7 of 1014] Compiling Type.Proxy
[   1 of 1014] Compiling Type.Equality
[   5 of 1014] Compiling Type.Row
[   6 of 1014] Compiling Web.Event.Internal.Types
[   2 of 1014] Compiling Spago.Texts
[   4 of 1014] Compiling Unsafe.Coerce
[   9 of 1014] Compiling Spago.Generated.BuildInfo
[  10 of 1014] Compiling Spago.Command.Script
[  11 of 1014] Compiling Record.Unsafe
[  12 of 1014] Compiling Partial
[  13 of 1014] Compiling JS.Fetch.Referrer
[  14 of 1014] Compiling Data.Void
[  15 of 1014] Compiling Data.Unit
[  16 of 1014] Compiling Data.Traversable.Accum
[  17 of 1014] Compiling Data.String.Unsafe
[  18 of 1014] Compiling Data.NaturalTransformation
[  19 of 1014] Compiling Data.Int.Bits
[  20 of 1014] Compiling Data.Boolean
[  21 of 1014] Compiling Data.ArrayBuffer.Types
[  22 of 1014] Compiling Control.Semigroupoid

  8. Interrupt the build.

  9. Record the statistics on output/Control.Semigroupoid/index.js.

    Access: 2024-01-25 01:19:46.011057433 +0300
Modify: 2024-01-25 01:35:40.739062556 +0300
Change: 2024-01-25 01:35:40.739062556 +0300
Birth: 2024-01-25 01:19:46.011057433 +0300

Expected result

f-f commented 10 months ago

Is this a spago issue? Does it behave differently if you just call purs? (if you run spago with -v it will print the purs commands it's calling under the hood)

deemp commented 10 months ago

IDK whether purs or spago is responsible for tracking the built files.

spago build -v shows that purs is called once and for all .purs files.

[       9ms] CWD: /home/eyjafjallajokull/Desktop/gh/spago
[      13ms] Global cache: "/home/eyjafjallajokull/.cache/spago-nodejs"
[      13ms] Local cache: "/home/eyjafjallajokull/Desktop/gh/spago/.spago"
[      47ms] DB: Connecting to database at /home/eyjafjallajokull/.cache/spago-nodejs/spago.v1.sqlite
[      50ms] Reading Spago workspace configuration...
[      50ms] Reading config from spago.yaml
[     147ms] Found packages at these paths:
  bin/spago.yaml
  core/spago.yaml
  docs-search/client-halogen/spago.yaml
  docs-search/common/spago.yaml
  docs-search/index/spago.yaml
[     148ms] Reading config from bin/spago.yaml
[     152ms] Reading config from core/spago.yaml
[     154ms] Reading config from docs-search/client-halogen/spago.yaml
[     158ms] Reading config from docs-search/common/spago.yaml
[     160ms] Reading config from docs-search/index/spago.yaml
[     362ms] Lockfile is up to date, using it
[     363ms] Found lockfile, using the package set from there
[     364ms] 
✅ Selecting 6 packages to build:
    docs-search-client-halogen
    docs-search-common
    docs-search-index
    spago
    spago-bin
    spago-core

[     364ms] Requested to install these packages: []
[     372ms] Downloading dependencies...
[     393ms] Building...
[     400ms] Running command:
purs compile --codegen corefn,docs,js,sourcemaps --json-errors .spago/p/aff-7.1.0/src/**/*.purs .spago/p/aff-promise-4.0.0/src/**/*.purs

...
f-f commented 10 months ago

Spago doesn't track builds, that's entirely happening in the compiler. Spago only puts together the globs and processes the output.

deemp commented 10 months ago

I see the compiler repo has a similar issue. I updated the description of this issue and will close it now.

MonoidMusician commented 10 months ago

The compiler won't reuse artefacts from canceled builds. You have to let a full build complete before you can see it skip compiling modules.