Explanation of incremental build feature

[timofei-iatsenko]

Hey, @folke thanks for the really good piece of software. The ultra-runner looks awesome and really promising for our workflow. Yesterday I ran it in our rush monorepo and as stated in docs it builds everything with zero-config, nice job!

However, I would like to know more about incremental build feature. To know how to optimize code or what patterns to avoid and what can cause optimization bailouts.

Could you elaborate a bit more on the strategy used for incremental build?

For example:

1. From doc:

   .ultra.cache.json file that contains hashes of all files and build artifacts in your repository. Internally this uses git ls-files for files under source control and simple mtime timestamps for build artifacts.

Wy we even should keep track on "build artefacts"? The build artefacts should not affect building, because they are, literally build artefacts, not sources.

Following up this question should I store my artefacts in project folder eq projects/app1/dist or it's better to hoist it into another level out of package folder?

2. What happened if I change root-level dependency or environment variable? For example root level tsconfig.json or dependencies in root level package.json. Does it treats package folder as "hermetic" where only changes applied to files inside that folder may produce different build no matter of environment (node version?).

3. More real-world example. I have a package with build command:

   "build": "gulp extract-messages && yarn build:bundle && yarn inline-messages",

   Where gulp extract-messages creates a *.pot file in {package-name}/src/translates.

I see on my CI that this invalidate cache every time. Means all consequent runs (without any changes) always trigger full rebuild.

As far as I understand ultra runner should create .ultra.cache.json after build command successfully executed. It means it should store mtime of these newly created files (kind of artefacts) and next time skips build.

But somehow these files invalidate cache every time. Do I understand the flow correctly? Do I miss something?

[remorses]

Regarding point 1, build artifacts means everything not tracked by git, these needs to be tracked in case the build result depends on these

To have ultra runner work on CI you will need to cache non tracked files (build artifacts, node moduels) and the .ultra.cache.json files, this way the next builds will skip unchanged packages

[folke]

The main reason we're tracking build artifacts is to make sure nothing was deleted in between builds, but to be fair, for most use cases this is probably overkill.

I don't fully understand why your project gets rebuild every time though. Are you certain nothing outside of build creates or modifies files? What is the list of changes that ultra shows?

[folke]

To answer your other questions:

1. another import reason to check node_modules is in a monorepo setup where your monorepo packages will be linked on your file system. Tracking those changes ensures a rebuild when another dependent package in your monorepo changes. (just remembered that this was the main reason I added this :) )
2. this won't trigger a rebuild, so you have to use --rebuild whenever you make those kinds of changes. ultra only looks within its own package and dependencies.

[timofei-iatsenko]

I don't fully understand why your project gets rebuild every time though. Are you certain nothing outside of build creates or modifies files? What is the list of changes that ultra shows?

I found the root cause of my issue. I had files that are "build artefacts", but was added (tracked) to the git. I don't remember why it was done like that, but adding them to git ignore and deleting from index fixed the issue.

Thanks for the answers, looks very good, now it works well.

[folke]

@thekip glad you were able to resolve your issue!