ndmitchell / rattle

Forward build system with speculation and caching
Other
102 stars 5 forks source link

performance improvements #19

Closed spall closed 4 years ago

spall commented 4 years ago

To improve Rattle's performance I replaced read/show with usage of the cereal library. In addition to the serialization changes I stored modtime with the tracing data to prevent the need to re-hash every file on a re-run.

These times are Rattle building vim when there is no work to be done on a rebuild with a single thread.

Before the changes: https://pastebin.com/VJjUAKKP (6 seconds)

After the serialization changes: https://pastebin.com/KcDBERSy (2.3 seconds)

After the serialization + hashing changes: https://pastebin.com/GzF5p3Lk (1.6 seconds)

spall commented 4 years ago

https://pastebin.com/43w9EgaX (1.3 seconds)

Reducing the amount of normalising decreased the running time some more; changes are in commit 99072eb

Although maybe this broke something on windows??

ndmitchell commented 4 years ago

The CI is failing on:

neil: The following files have \r characters, trailing whitespace or excess newlines.
/tmp/extra-dir-55775753346261/rattle-0.1/rattle.cabal
/tmp/extra-dir-55775753346261/rattle-0.1/src/Development/Rattle/Hash.hs
/tmp/extra-dir-55775753346261/rattle-0.1/src/Development/Rattle/Shared.hs
/tmp/extra-dir-55775753346261/rattle-0.1/src/General/FileInfo.hs
/tmp/extra-dir-55775753346261/rattle-0.1/src/General/FileName.hs

Can you make sure you save those with Unix line endings? That should let the CI get further and we can see where things stand.

Something does seem broken on Windows, but I use Windows, so I can debug/diagnose that after we merge easily enough. It doesn't seem easily fixable only with the CI output.

spall commented 4 years ago

I squashed some of these bug fix commits hope that is ok; they were getting excessive.

ndmitchell commented 4 years ago

Thanks, looks good! I'll fix up the remaining Windows niggle. Happy to have squashed commits or not - not too fussed either way.