search

jfrog / artifactory-maven-plugin

A Maven plugin to resolve artifacts from Artifactory, deploy artifacts to Artifactory, capture and publish build info.
https://www.jfrog.com/confluence/display/JFROG/Maven+Artifactory+Plugin
Apache License 2.0
24 stars 26 forks source link

Assuming reproducible builds, how about a do-not-fail-if-already-published config option #7

Closed paul-hammant closed 5 months ago

paul-hammant commented 3 years ago

Is your feature request related to a problem? Please describe.

Context: Industry problem of how to keep release branches and CI forcing the maintainer of the release branch to update the version numbers of the modules that have change but not all versions numbers for the set of modules at the same time

Context: Refer: https://paulhammant.com/2018/05/23/examining-ci-cd-and-branching-models/

Context: Assuming https://maven.apache.org/guides/mini/guide-reproducible-builds.html for a Maven project in source control.

If a passing build with a so-configured artifactory-maven-plugin were to attempt to publish an artifact, it could first check to see whether the GAV has been published already. If it has and the SHA1 for the pom and the binary are the same, then it could simply pass that step without uploading the pom/binary.

This would be optional. Without do-not-fail-if-already-published=true the build would fail as it does today as the artifact is already published.

To reiterate:

  1. with do-not-fail-if-already-published=false (or the property is absent) then the build continues to fail if the GAV is published.
  2. with do-not-fail-if-already-published=true then the build passes the GAV is not yet published
  3. with do-not-fail-if-already-published=true then the build passes the GAV is already published and it has the same hashes (the actual upload is silently skipped)
  4. with do-not-fail-if-already-published=true then the build fails the GAV is already published and it has the different hashes

Describe alternatives you've considered

Asking for changes to Artifactory itself - changes to the plugin are simpler. And if JFrog say no, it can always be forked at the optional feature added anyway :-O

Additional context

Reproducible builds are an industry quest.

If achieved, there's a positive consequence for binary repositories everywhere - all languages & all binary repo idioms.

Caveats

I should never be allowed to name anything, so change do-not-fail-if-already-published to your preferred english expression.

yahavi commented 3 years ago

@paul-hammant, The Maven Artifactory plugin supports checksum deployment, which deploys only if the artifact does not exist in Artifactory. This feature is meant to be a performance boost for uploads. If I understood it right, using this feature, may help you. Artifacts with the same checksum are not uploaded to Artifactory.

By default, this feature is off for small files, below 10KB. In the new version, 3.2.0, we added an option to configure this threshold.

In order to enable checksum deploy for all files please do the following:

  1. Upgrade Maven Artifactory plugin to 3.2.0.
  2. Set minChecksumDeploySizeKb to 0: 
    <configuration>
<publisher>
<minChecksumDeploySizeKb>0</minChecksumDeploySizeKb>
</<publisher>
</<configuration>

You can read more in a similar issue we resolved for Gradle: https://github.com/jfrog/build-info/issues/341 Please let us know if it helped.

paul-hammant commented 3 years ago

Sounds like what I want - I've posed a question on that issue

paul-hammant commented 3 years ago

I can use artifactory-maven-plugin for any WebDAV-capable maven repo technology, right? Meaning apart from anything else artifactory-maven-plugin is a solid publisher of artifacts, and viable general purpose alternative to the artifact publishing inside Maven itself ?

yahavi commented 3 years ago

@paul-hammant, I'm not sure what does it mean by "Maven itself". Maven is a package manager, not a binary repository.

The Artifactory Maven plugin is, as it sounds, a plugin for Artifactory repositories. Its goal is to enforce deployment to Artifactory, as well as collect and publish the build info. The plugin replaces the default Maven deployment process with a deployment that collects the build info and uploads Artifact to Artifactory. Publishing to Maven repositories that are not Artifactory will probably not work.

Please let me know if it answered your question. If not, please elaborate.

jakobbp commented 3 years ago

Just a heads-up to anyone whom might not have thought about it: make sure you use Maven plugins that support reproducible builds: https://maven.apache.org/guides/mini/guide-reproducible-builds.html Otherwise checksum comparison will always fail, since Maven prints build timestamp into pom.properties, which is found inside all artifacts generated by the default plugins. Without these preparations, checksum comparison will only work for POM packaging deployments.