Closed wwilk closed 6 years ago
Can you write an overview of the change. I have a couple of questions :) In general, I'm looking for every opportunity to simplify the code.
Can you make sure the feature is documented at the point of merging the PR? I think that we should start adhering to the principle that every PR has documentation. I'm documenting for 1.0 release and it is a pain to do it afterwards :)
I think that better home for createDependencyInfoFile task is ComparePublicationsPlugin instead of JavaLibraryPlugin. This way, we can document how it works in ComparePublicationsPlugin javadoc. Long term, I think that our plugins should be documented in the Javadoc rather than in markdown docs. This way, docs live closer to the code and it easier to keep it up to date (and stay motivated to do so).
Why do we need to write any code to parse the dependency-info.json file? If we ensure that the dependency-info.json contents are written in a consistent way we can leverage an existing comparison / diff logic. I'd hope that we can reduce the complexity as much as possible and avoid writing a parser / comparator for the file.
Do we need to introduce more zip parsing logic? I thought that we already have all the code in place for that. In the PR it looks like new zip logic was introduced. Again, it seems to me that extra complexity can be avoided. We can also use Gradle's zip features to avoid maintaining our own code.
How do you avoid version problem for internal (submodule) dependencies? Do you add secret souse to the parser/filter when we compare the json file? Given that you have tested it with release-example this must work with submodule dependencies (cool!!!).
dependency-info.json looks untidy with an empty artifact and is inconsistent with Gradle model. AFAIR, Gradle returns 'null' when there is no artifact. Let's change the metadata format to something like this:
{
"artifact": null,
"name":"diffutils",
"version":"1.3.0",
"group":"com.googlecode.java-diff-utils"
}
To simplify comparison, perhaps we want to avoid JSON and have a flat list / yml. This way, we don't ever parse the file, we diff it textually as every other file in the sources jar. Examples:
description: xxx
dependencies:
# standard dependencies:
just text:
#description xxx
#dependencies
com.googlecode.java-diff-utils:diffutils:1.3.0
com.gradle.publish:plugin-publish-plugin:0.9.7
#internal dependency, no version
foo:bar
#dependency with artifact
foo:bar:1.0
#name:classifier:type:extension notation for artifacts
foo:sources:jar:jar
BTW, the code is greatly simplified when we don't have to parse the POM. BIG THANKS for working on this!!!
OK, I changed the format of dependency-info and created a separate PR for that #543, to keep it simpler. When it's merged I will update this one.
Dependency-info.md now used.
Example output in shipkit when added one dependency and changed version of another:
Differences between files:
--- /Users/wilk/Projects/shipkit/subprojects/shipkit/build/previous-release-artifacts/shipkit-0.9.130-sources.jar
+++ /Users/wilk/Projects/shipkit/subprojects/shipkit/build/libs/shipkit-0.9.131-sources.jar
Added files:
++ org/shipkit/internal/comparison/DependencyInfoComparator.java
++ org/shipkit/internal/gradle/util/ZipUtil.java
Removed files:
-- org/shipkit/internal/comparison/PomComparator.java
Modified files:
+- org/shipkit/gradle/java/ComparePublicationsTask.java
+- org/shipkit/gradle/java/DownloadPreviousPublicationsTask.java
+- org/shipkit/internal/comparison/ZipComparator.java
+- org/shipkit/internal/comparison/diff/Diff.java
+- org/shipkit/internal/comparison/diff/FileDiffGenerator.java
+- org/shipkit/internal/gradle/java/ComparePublicationsPlugin.java
+- org/shipkit/internal/gradle/java/tasks/ComparePublications.java
+- org/shipkit/internal/gradle/java/tasks/DownloadPreviousPublications.java
Differences between files:
--- /Users/wilk/Projects/shipkit/subprojects/shipkit/build/previous-release-artifacts/shipkit-0.9.130-sources.jar/META-INF/dependency-info.md
+++ /Users/wilk/Projects/shipkit/subprojects/shipkit/build/libs/shipkit-0.9.131-sources.jar/META-INF/dependency-info.md
Here you can see the changes in declared dependencies between versions.
@@ -7,0 +7,1 @@
+ - com.googlecode.json-simple:json-simple:1.1.1^M
@@ -8,1 +9,1 @@
- - com.jfrog.bintray.gradle:gradle-bintray-plugin:1.7.3
+ - com.jfrog.bintray.gradle:gradle-bintray-plugin:1.7.1
I didn't change anything about comparing sources jars, but still here for you to see the whole picture.
About the zip complexity - I extracted typical zip operations into ZipUtil class. It's a little harder because I would like to check in comparePublications if dependency-info.md is present in previous sources jar, to keep the backwards compatibility. For this I have to open the zip file.
I have to open it in two other places - ZipComparator and DependencyInfoComparator, because I need to extract dependency-info.md from jar. I don't like passing around open zip file, so I prefer to open and parse it in a few different places, even if it has a slight performance impact.
About the zip complexity - I extracted typical zip operations into ZipUtil class. It's a little harder because I would like to check in comparePublications if dependency-info.md is present in previous sources jar, to keep the backwards compatibility. For this I have to open the zip file.
hmmm, I suspect that we can simplify the implementation. For example, during zip comparison you can notify some listener that some file is different in zips and here're the input streams. The listener implementation could do content comparison for clean diff.
In general, I'd shy away from creating "special" implementations for diffing a particular file from the zip.
I'm tempted to skip dependency-info comparison. We don't do it for other files in the jar, why do we bother for this one? However, since we already have the code, let's keep it because it can potentially improve developer experience in some scenarios.
BTW. the output looks amazing!!! Thank you!
I created a new class ComparePublicationsResultFormatter and moved there all the complexity regarding the above comparison output. Also instead of DependencyInfoComparator we have a generic StringComparator
We can now use dependency-info.json for comparing publications. The file started to be generated in Shipkit 0.9.122.
Tested in shipkit and shipkit-example by running "./gradlew assertReleaseNeeded".