search

i2p / i2p.i2p

I2P is an anonymizing network, offering a simple layer that identity-sensitive applications can use to securely communicate. All data is wrapped with several layers of encryption, and the network is both distributed and dynamic, with no trusted parties.
https://geti2p.net
Other
1.95k stars 305 forks source link

Improve GRADLE build Performance #23

Closed shisheng-1 closed 2 years ago

shisheng-1 commented 2 years ago

Parallel builds. This project contains multiple modules. Parallel builds can improve the build speed by executing tasks in parallel. We can enable this feature by setting org.gradle.parallel=true.

Configuration on demand. Configuration on demand tells Gradle to configure modules that only are relevant to the requested tasks instead of configuring all of them. We can enable this feature by setting org.gradle.configureondemand=true.

gradle caching. Shared caches can reduce the number of tasks you need to execute by reusing outputs already generated elsewhere. This can significantly decrease build times. We can enable this feature by setting org.gradle.caching=true.

===================== If there are any inappropriate modifications in this PR, please give me a reply and I will change them.

zlatinb commented 2 years ago

While I was working on making the Gradle builds reproducible I ran into some problems with parallel builds. Can you confirm that you always get the exact same files in 

reproducible/build/distributions

with parallel build enabled?

shisheng-1 commented 2 years ago

While I was working on making the Gradle builds reproducible I ran into some problems with parallel builds. Can you confirm that you always get the exact same files in

reproducible/build/distributions

with parallel build enabled?

Thank you @zlatinb .Actually, I hadn't thought about it before.Because the principle of Parallel builds is that for multi-module projects, you can take full advantage of multiple processors and build them at the same time. I think that if the build is successful, the result should be reasonable, and the order of the build should not cause errors

I wonder what you think of the other two configurations.

Have a nice day!

shisheng-1 commented 2 years ago

Hi @zlatinb! We use ./gradlew clean build -x test --scan to get the build time

zlatinb commented 2 years ago

@shisheng-1 I need to verify that the outputs inside the reproducible subproject are indeed reproducible. If they are, I'm ok with merging this PR

@eyedeekay will merging this PR propagate to GitLab automatically?

eyedeekay commented 2 years ago

@zlatinb It should sync up automatically, but we haven't tried it yet. Let me do the merge and reconcile any issues. Ping me when I have your ack and I'll make sure everything goes just well. To clarify, the sync bot pulls in every change on both masters and merges them continuously, it should not be possible for a merge conflict to arise unless a push which conflicts comes in on the other mirror within about a minute of this getting merged.

I'll attempt a reproducible build shortly and post a hash.

eyedeekay commented 2 years ago

4ffe9ff27d1fcbb7687646e2229f26ca593d213d4a995e41b8fe4af77f7969e0 reproducible/build/distributions/i2p.zip

Here's my hash made from the tip of shisheng-1's branch

eyedeekay commented 2 years ago

Forgot to mention, that's with java 1.8.0

eyedeekay commented 2 years ago

@zlatinb What's the word here? Should I merge?

zlatinb commented 2 years ago

No, I'm getting 6471a60712874959e0642e8e32844ac96f54008716f6fba3549ac2865a0d0517 from the tip of the Modify_GRADLE_1 branch. However, I am getting that in a reproducible fashion with ant distclean in between.

I don't have the capacity to look into it as I'm away from my main workstation.

eyedeekay commented 2 years ago

@zlatinb when I do an ant distclean and a ./gradlew clean assemble using Java 11 our hashes match. I think our hashes were different only because I had update-java-alternatives configured to point to Java 8.

6471a60712874959e0642e8e32844ac96f54008716f6fba3549ac2865a0d0517 reproducible/build/distributions/i2p.zip

zlatinb commented 2 years ago

Sorry I missed the note about Java 1.8. Yes please merge.