developersu / ns-usbloader

Awoo Installer and GoldLeaf uploader of the NSPs (and other files), RCM payload injector, application for split/merge files.
GNU General Public License v3.0
1.82k stars 123 forks source link

feat: make pom.xml reproducible #150

Open soupglasses opened 1 year ago

soupglasses commented 1 year ago

Following packaging ns-usbloader to NixOS I did some work on making maven generate reproducible jar files following the advice on https://maven.apache.org/guides/mini/guide-reproducible-builds.html

I have also added maven enforcer to ensure that all maven plugins used are properly defined in the pom.xml file.

Only major issue was that i was unable to find a good practice for was how to include javafx without making it architecture dependent. There are also a long set of warnings about its inability to build.

[WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-base:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-base:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1
[WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1
Verify reproducible artifact log ```console mvn clean verify artifact:compare Picked up _JAVA_OPTIONS: -Djava.util.prefs.userRoot=/home/sofi/.config/java [INFO] Scanning for projects... [INFO] [INFO] -------------------------< loper:ns-usbloader >------------------------- [INFO] Building NS-USBloader 7.0 [INFO] from pom.xml [INFO] --------------------------------[ jar ]--------------------------------- [INFO] [INFO] --- clean:3.3.1:clean (default-clean) @ ns-usbloader --- [INFO] Deleting /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target [INFO] [INFO] --- enforcer:3.3.0:enforce (enforce-versions) @ ns-usbloader --- [INFO] Rule 0: org.apache.maven.enforcer.rules.version.RequireMavenVersion passed [INFO] Rule 1: org.apache.maven.enforcer.rules.version.RequireJavaVersion passed [INFO] Rule 2: org.apache.maven.enforcer.rules.RequirePluginVersions passed [INFO] [INFO] --- resources:3.3.1:resources (default-resources) @ ns-usbloader --- [INFO] Copying 61 resources from src/main/resources to target/classes [INFO] Copying 1 resource from src/main/resources-filtered to target/classes [INFO] The encoding used to copy filtered properties files have not been set. This means that the same encoding will be used to copy filtered properties files as when copying other filtered resources. This might not be what you want! Run your build with --debug to see which files might be affected. Read more at https://maven.apache.org/plugins/maven-resources-plugin/examples/filtering-properties-files.html [INFO] [INFO] --- compiler:3.11.0:compile (default-compile) @ ns-usbloader --- [INFO] Changes detected - recompiling the module! :source [INFO] Compiling 99 source files with javac [debug release 11] to target/classes [INFO] [INFO] --- resources:3.3.1:testResources (default-testResources) @ ns-usbloader --- [INFO] skip non existing resourceDirectory /mnt/home/sofi/Code/Github/developersu/ns-usbloader/src/test/resources [INFO] [INFO] --- compiler:3.11.0:testCompile (default-testCompile) @ ns-usbloader --- [INFO] Changes detected - recompiling the module! :dependency [INFO] Compiling 4 source files with javac [debug release 11] to target/test-classes [INFO] [INFO] --- surefire:3.1.2:test (default-test) @ ns-usbloader --- [INFO] Using auto detected provider org.apache.maven.surefire.junitplatform.JUnitPlatformProvider [INFO] [INFO] ------------------------------------------------------- [INFO] T E S T S [INFO] ------------------------------------------------------- Picked up _JAVA_OPTIONS: -Djava.util.prefs.userRoot=/home/sofi/.config/java [INFO] Running nsusbloader.com.usb.SplitTest I: Split files: [0] TestFile4Split_0 [1] TestFile4Split_1 [2] TestFile4Split_2 [3] TestFile4Split_3 [4] TestFile4Split_4 [5] TestFile4Split_5 [6] TestFile4Split_6 I: [2] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_2 I: [4] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_4 I: [1] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_1 I: [3] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_3 I: [6] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_6 I: [0] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_0 I: [5] Save results to: /tmp/junit2117474296429378177/!_TestFile4Split_5 I: [6] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_6 (size: 9295) I: [2] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_2 (size: 14993) I: [4] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_4 (size: 14654) I: [1] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_1 (size: 12956) I: [5] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_5 (size: 12818) I: [5] Chunks 00 size: 12818 Total chunks size: 12818 I: [0] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_0 (size: 11595) I: [2] Chunks 00 size: 14993 Total chunks size: 14993 I: [4] Chunks 00 size: 14654 Total chunks size: 14654 I: [0] Chunks 00 size: 11595 Total chunks size: 11595 P: [5] Sizes are the same! Split file should be good! P: [2] Sizes are the same! Split file should be good! P: [0] Sizes are the same! Split file should be good! P: [4] Sizes are the same! Split file should be good! I: [1] Chunks 00 size: 12956 Total chunks size: 12956 I: [6] Chunks 00 size: 9295 Total chunks size: 9295 P: [1] Sizes are the same! Split file should be good! I: [3] Original file: /tmp/junit2117474296429378177/!_TestFile4Split_3 (size: 11136) P: [6] Sizes are the same! Split file should be good! I: [3] Chunks 00 size: 11136 Total chunks size: 11136 P: [3] Sizes are the same! Split file should be good! I: .:: Split complete ::. [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.104 s -- in nsusbloader.com.usb.SplitTest [INFO] Running nsusbloader.com.usb.MergeTest I: Merge files: [0] TestFile4Merge_0 [1] TestFile4Merge_1 [2] TestFile4Merge_2 [3] TestFile4Merge_3 [4] TestFile4Merge_4 [5] TestFile4Merge_5 [6] TestFile4Merge_6 I: [0] Next files will be merged in following order: 00 01 02 03 04 05 06 I: [0] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_0 I: [1] Next files will be merged in following order: 00 01 02 03 04 05 I: [1] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_1 I: [2] Next files will be merged in following order: 00 01 02 03 I: [2] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_2 I: [4] Next files will be merged in following order: 00 01 I: [4] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_4 I: [3] Next files will be merged in following order: 00 01 02 I: [3] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_3 I: [5] Next files will be merged in following order: 00 01 02 03 04 I: [5] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_5 I: [6] Next files will be merged in following order: 00 01 02 I: [6] Save results to: /tmp/junit15201410954350906640/!_TestFile4Merge_6 I: [2] Total chunks size: 57292 Merged file size: 57292 P: [2] Sizes are the same! Resulting file should be good! I: [4] Total chunks size: 20568 Merged file size: 20568 P: [4] Sizes are the same! Resulting file should be good! I: [1] Total chunks size: 51966 Merged file size: 51966 P: [1] Sizes are the same! Resulting file should be good! I: [0] Total chunks size: 107758 Merged file size: 107758 P: [0] Sizes are the same! Resulting file should be good! I: [3] Total chunks size: 41064 Merged file size: 41064 P: [3] Sizes are the same! Resulting file should be good! I: [6] Total chunks size: 28677 Merged file size: 28677 P: [6] Sizes are the same! Resulting file should be good! I: [5] Total chunks size: 48160 Merged file size: 48160 P: [5] Sizes are the same! Resulting file should be good! I: .:: Merge complete ::. [INFO] Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.037 s -- in nsusbloader.com.usb.MergeTest [INFO] Running nsusbloader.com.usb.TransferModuleTest [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.016 s -- in nsusbloader.com.usb.TransferModuleTest [INFO] [INFO] Results: [INFO] [INFO] Tests run: 5, Failures: 0, Errors: 0, Skipped: 0 [INFO] [INFO] [INFO] --- assembly:3.6.0:single (make-assembly) @ ns-usbloader --- [WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-base:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-base:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-graphics:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-controls:jar:19.0.2.1 [WARNING] Failed to build parent project for org.openjfx:javafx-fxml:jar:19.0.2.1 [INFO] Artifact ru.redrise:libKonogonka:pom:0.1 is present in the local repository, but cached from a remote repository ID that is unavailable in current build context, verifying that is downloadable from [central (https://repo.maven.apache.org/maven2, default, releases)] [INFO] Building jar: /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/ns-usbloader-7.0.jar [WARNING] Configuration option 'appendAssemblyId' is set to false. Instead of attaching the assembly file: /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/ns-usbloader-7.0.jar, it will become the file for main project artifact. NOTE: If multiple descriptors or descriptor-formats are provided for this project, the value of this file will be non-deterministic! [WARNING] Replacing pre-existing project main-artifact file: /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/classes with assembly file: /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/ns-usbloader-7.0.jar [INFO] [INFO] --- artifact:3.4.1:compare (default-cli) @ ns-usbloader --- [WARNING] No source information available in buildinfo for rebuilders... [INFO] Saved info on build to /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/ns-usbloader-7.0.buildinfo [INFO] Checking against reference build from central... [INFO] Reference buildinfo file not found: it will be generated from downloaded reference artifacts [INFO] Reference build java.version: 19 (from MANIFEST.MF Build-Jdk-Spec) [INFO] Minimal buildinfo generated from downloaded artifacts: /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/reference/ns-usbloader-7.0.buildinfo [INFO] Reproducible Build output summary: 2 files ok [INFO] Reproducible Build output comparison saved to /mnt/home/sofi/Code/Github/developersu/ns-usbloader/target/ns-usbloader-7.0.buildcompare [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 7.159 s [INFO] Finished at: 2023-08-30T10:57:33+02:00 [INFO] ------------------------------------------------------------------------ ```
soupglasses commented 1 year ago

From my understanding is that how JavaFX is currently packaged in this project what is the problem. Sadly my Java experience is limited, but i did find https://stackoverflow.com/a/70809214 if this is something that is interesting. Then the jar could be architecture independent from my understanding.

developersu commented 1 year ago

Thank you for contributing!

Only major issue was that i was unable to find a good practice for was how to include javafx without making it architecture dependent. There are also a long set of warnings about its inability to build.

Yeah.. same. Well, it is what it is :(

From my understanding is that how JavaFX is currently packaged in this project what is the problem. Sadly my Java experience is limited, but i did find https://stackoverflow.com/a/70809214 if this is something that is interesting. Then the jar could be architecture independent from my understanding.

I don't know everything either :) Main idea of this stackoverflow post 'Don't try to create a single jar for your application.' as I see it. So a workaround for Linux is making some kind of java-openjfx dependency and do not include it into fat-JAR, for Windows it's possible to assembly jar-runtime (via jlink) from some JDK and deploy it via installer. As for Mac I have no idea (and no Mac). In the end ignoring warnings looks like an easiest solution.

~Also as for set of plugins you added, do you think it's really needed? Like 'maven-resources-plugin' and others?~ Ok, I see :D

developersu commented 1 year ago

@soupglasses please check this once you have a minute. Wanna know your thoughts. Thanks!

soupglasses commented 1 year ago

Will do :)