search

folke / ultra-runner

🏃⛰ Ultra fast monorepo script runner and build tool
https://www.npmjs.com/package/ultra-runner
Apache License 2.0
1.2k stars 32 forks source link

Idea: more efficient build strategy #178

Open lxgreen opened 3 years ago

lxgreen commented 3 years ago

Hey ultra people,

Thanks for this cool piece of code!

I'd like to share an idea. As far as I understand, currently ultra cache relies on git file hash for change detection in files, and timestamps for change detection in directories. I can see, the cache does not include the build artifact caches (e.g. in my project, the cache contains an entry for dist directory, but not for its contents).

What if the cache would contain entries for build artifacts as well, while their hashes are affected by the contents (e.g. md5 checksum)? This would provide actual change detection. Say, I've just changed a typescript annotation or a comment in package A. This change triggers package A rebuild. After the package is rebuilt, we compare bundle checksums vs cached ones. Since the change is dev-time only, it doesn't affect the bundle contents, so the rebuild can stop here and the dependent packages don't have to be rebuilt.

WDYT?

folke commented 3 years ago

I thought about something like that when I implemented ultra, but having to calculate checksums for all build artifacts might slow-down the builds quite a bit.

Triggering a rebuild on mtime changes of those directories should work for most cases and only sometimes do a build that night not have been needed.

lxgreen commented 3 years ago

Agree on that, in general. Hopefully, I'll find time to make POC to see the checksum calculation impact. This calculation supposed to run after the build is complete, so if it could run somehow in the background in non-blocking manner... 🤔