search

alexo / wro4j

New project location is:https://github.com/wro4j/wro4j
442 stars 110 forks source link

Optimize how often files are checksummed #202

Closed muffl0n closed 10 years ago

muffl0n commented 10 years ago

In #189 we already discussed that persistingFingerprint was taking more time than it should. I looked into this and found that files that are included by many different files get fingerprinted every time. The problem is that you can't just ask the BuildContextHolder if it has a value for the key (URI of the resource) because it would return the value for this key stored in a former build.

So I added a set to BuildContextHolder that stores if the key was set in the current run. This set is not persisted so it only contains keys that are recently set. ResourceChangeHandler then asks the BuildContextHolder if the value (checksum) for the key (URI of the resource that should be fingerprinted) has already been calculated in this run. If this is true it just skips the costly fingerprinting of said resource.

This way we save a lot of time in incrementalBuilds. We got our building time down 84% to 2s from 13s without the optimization.

I'm not quite happy with the wording though. So this is more a prove of concept that works remarkably well with our setup. Yes, we have a lot of includes and cascaded files. :)

alexo commented 10 years ago

I will merge this PR in a different branch. There is still a test that fails and probably I'll apply small changes before merging it.

muffl0n commented 10 years ago

Shame on me, I run "mvn install" with "-Dmaven.test.skip=true". :( The problem is that the tests that try to connect to webservers fail on my machine because I'm behind a very restrictive proxy at work. Didn't figure out how to solve this yet. But that is definitely on my list.

muffl0n commented 10 years ago

Thank you! :+1: