Redeployed snapshots ignored.

[saundersjr]

Hello,

I'm allowing redeploy on my nexus maven snapshot repository. If I deploy an new version of a snapshot (1.0.23-SNAPSHOT), the go-maven-poller notices this and my Go pipeline is triggered. This is expected behavior. However if I redeploy the same snapshot version (1.0.23-SNAPSHOT), I have redploy on, the go-maven-poller doesn't trigger the pipeline.

2016-07-15 12:14:01,755 INFO [82@MessageListener for MaterialUpdateListener] RepositoryClient:52 - lastKnownVersion is 1.0.23-SNAPSHOT 2016-07-15 12:14:01,755 INFO [82@MessageListener for MaterialUpdateListener] RepositoryClient:52 - no newer version 2016-07-15 12:14:01,756 WARN [82@MessageListener for MaterialUpdateListener] RepositoryClient:62 - getLatest returning null 2016-07-15 12:14:01,756 INFO [82@MessageListener for MaterialUpdateListener] MavenRepositoryPoller:52 - no modification since 1.0.23-SNAPSHOT

Is there any chance of allowing redeployed snapshots to act as a trigger?

[thunder-spb]

Having the same issue :(

[bitionaire]

Hi guys,

thank you for reporting the issue!

Basically this poller processes the maven-metadata.xml files of any repository and for SNAPSHOT versions it tries to parse XPaths for both /metadata/versioning/snapshot/timestamp and /metadata/versioning/snapshot/buildNumber in order to create a "new", more specific version number. Please have a look if those elements are set within the directory of the SNAPSHOT.

I'll try to debug it by tomorrow and get back to you as soon as I have more answers.

Cheers

[bitionaire]

Hi,

you were right, there's a bug, because the comparison of the latest revision and the last processed revision does not include the timestamp or build number. These information are only used to calculate the correct download location (URL) of the artifact.

In order to fix this problem the revision response of this plugin must include the timestamp and build number for SNAPSHOT versions. Once the latest revision will be reported by the Go Server, this information must be reconstructed, in order to be used for comparison. Caveats: The Go UI will display the modified version String instead of the original one defined in the pom.xml. E.g. 1.2-SNAPSHOT (201601234...) will be displayed instead of 1.2-SNAPSHOT.

Sorry @saundersj that I didn't respond earlier and your issue was idling for 24 days. I didn't notice the GitHub mail notification. I'll try to fix that issue for you, possibly today.

Cheers

[thunder-spb]

Hi @bitionaire . Cool, thanks!

[saundersjr]

That's great that you have a fix coming.

No problem about the delay in getting back. I appreciate you finding a potential fix.

Let me know when you are ready, I can run some tests against our snapshot builds.

On 9 Aug 2016 10:20, "Johann Böhler" notifications@github.com wrote:

Hi,

you were right, there's a bug, because the comparison of the latest revision and the last processed revision does not include the timestamp or build number. These information are only used to calculate the correct download location (URL) of the artifact.

In order to fix this problem the revision response of this plugin must include the timestamp and build number for SNAPSHOT versions. Once the latest revision will be reported by the Go Server, this information must be reconstructed, in order to be used for comparison. Caveats: The Go UI will display the modified version String instead of the original one defined in the pom.xml. E.g. 1.2-SNAPSHOT (201601234...) will be displayed instead of 1.2-SNAPSHOT.

Sorry @saundersj https://github.com/saundersj that I didn't respond earlier and your issue was idling for 24 days. I didn't notice the GitHub mail notification. I'll try to fix that issue for you, possibly today.

Cheers

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/1and1/go-maven-poller/issues/9#issuecomment-238499531, or mute the thread https://github.com/notifications/unsubscribe-auth/AGmNsZlx7VodVg2zsDHyi_BAAz2jJ67fks5qeEZsgaJpZM4JNV_8 .

[bitionaire]

The issue might be fixed in pre-release 1.1.1, but I couldn't test it yet because of a missing test environment here at home. It would take me quite some time to set it up (Maven repo, Go Server, Configuration of the Pipelines etc.). I know it's not your job, but can you probably test it - with and without SNAPSHOT versions? Would save me a lot of time.

If your test will fail, then I promise that I'll setup everything I need and prepare a fix for you asap. Deal? 😃

[saundersjr]

Yes no problem. I have a test rig with snapshot pipelines already setup. I just need to install the latest poller jar. I will hopefully test it tomorrow. Speak soon.

On 9 Aug 2016 7:22 p.m., "Johann Böhler" notifications@github.com wrote:

The issue might be fixed in pre-release 1.1.1 https://github.com/1and1/go-maven-poller/releases/tag/1.1.1, but I couldn't test it yet because of a missing test environment here at home. It would take me quite some time to set it up (Maven repo, Go Server, Configuration of the Pipelines etc.). I know it's not your job, but can you probably test it - with and without SNAPSHOT versions? Would save me a lot of time.

If your test will fail, then I promise that I'll setup everything I need and prepare a fix for you asap. Deal? 😃

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/1and1/go-maven-poller/issues/9#issuecomment-238644001, or mute the thread https://github.com/notifications/unsubscribe-auth/AGmNsQtKWF_q_hYv1C4IvLuyt8VEAhGIks5qeMVWgaJpZM4JNV_8 .

[thunder-spb]

@bitionaire didn't fix that issue for me unfortunately...

[bitionaire]

Ok, I'm on it

[thunder-spb]

@bitionaire if you need details feel free to ask :)

[bitionaire]

Please try again. It worked for me with Maven Central & Snapshot repositories.

There may be some problems as there's no complete schema for the maven-metadata.xml file and I'm relying on the time format I have seen so far (YYYYMMDD.HHmmss) specified for the timestamp element. This is what I've observed so far for Artifactory repositories and the Maven central.

[thunder-spb]

That is my metadata file:

<metadata modelVersion="1.1.0">
<groupId>com.sample.group</groupId>
<artifactId>sample-artifact</artifactId>
<version>1.0.273-SNAPSHOT</version>
<versioning>
<snapshot>
<timestamp>20160810.094448</timestamp>
<buildNumber>2</buildNumber>
</snapshot>
<lastUpdated>20160810094448</lastUpdated>
<snapshotVersions>
<snapshotVersion>
<extension>jar</extension>
<value>1.0.273-20160810.094448-2</value>
<updated>20160810094448</updated>
</snapshotVersion>
<snapshotVersion>
<extension>pom</extension>
<value>1.0.273-20160810.094448-2</value>
<updated>20160810094448</updated>
</snapshotVersion>
<snapshotVersion>
<extension>zip</extension>
<value>1.0.273-20160810.094448-2</value>
<updated>20160810094448</updated>
</snapshotVersion>
</snapshotVersions>
</versioning>
</metadata>

That is second artifact for this version, but poller gets first artifact, at least when I run it manually

[thunder-spb]

11:21:35.665 [go] setting environment variable 'GO_PACKAGE_NEXUS_SNAPSHOTS_NEW_SAMPLE_ARTIFACT_VERSION' to value '1.0.273-20160808.172358-1'

[bitionaire]

Ok this is strange.. I tested it with the artifact snmpman and the pipeline was triggered as expected once I deployed a new SNAPSHOT version.

I set the pipeline label to the material so you can see the version increase actually.

Are you sure this isn't related to some repositories which aren't in sync and the load balancer directs you and your GoCD server to some different locations?

[thunder-spb]

hm, just uploaded new snapshot build and nothing. looking into plugin-maven-repo.log: 2016-08-10 12:07:20,394 INFO [89@MessageListener for MaterialUpdateListener] RepositoryClient:52 - lastKnownVersion is 1.0.273-SNAPSHOT 2016-08-10 12:07:20,394 INFO [89@MessageListener for MaterialUpdateListener] RepositoryClient:52 - no newer version 2016-08-10 12:07:20,394 WARN [89@MessageListener for MaterialUpdateListener] RepositoryClient:62 - getLatest returning null 2016-08-10 12:07:20,394 INFO [89@MessageListener for MaterialUpdateListener] MavenRepositoryPoller:52 - no modification since 1.0.273-SNAPSHOT

And we don't have any loadbalancers infront of nexus. But when I'm clicking "Check connection" is see new version...

[thunder-spb]

Just in case, GoCD version is 16.7, I've deleted old (aserok) maven-poller plugin, didn't cleanup db...

[bitionaire]

Ok, unfortunately I had the same problem initially and had to delete and recreate the package configuration and the pipeline (or the pipeline history). I thought this issue might be fixed by following code in the MavenVersion#compareTo(..) method

if (otherVersion.timestamp == null && otherVersion.buildNumber == null) {
    result = 1;
}
if (this.timestamp == null && this.buildNumber == null) {
    result = -1;
}

but as it seems that it doesn't, I'll have to further investigate.

[bitionaire]

I attached an updated version to release 1.1.1. Please download the new go-maven-poller.jar, copy it to your GoCD's server plugins/external directory and restart the GoCD server. If this won't fix the problem than I don't know what will.

[thunder-spb]

Great job! Works now!

2016-08-10 13:08:11,279 INFO [96@MessageListener for MaterialUpdateListener] RepositoryClient:52 - set snapshot information to specific version 1.0.273-SNAPSHOT (20160810.130649-4) 2016-08-10 13:08:11,281 INFO [96@MessageListener for MaterialUpdateListener] RepositoryClient:52 - lastKnownVersion is 1.0.273-SNAPSHOT (20160810.120216-3) 2016-08-10 13:08:11,282 INFO [96@MessageListener for MaterialUpdateListener] RepositoryClient:52 - Latest is 1.0.273-SNAPSHOT (20160810.130649-4) 2016-08-10 13:08:11,681 INFO [96@MessageListener for MaterialUpdateListener] MavenRepositoryPoller:52 - latest revision is 1.0.273-SNAPSHOT (20160810.130649-4)

[thunder-spb]

And you need to add "space" after "lastKnownVersion" word: 2016-08-10 13:06:41,517 INFO [92@MessageListener for MaterialUpdateListener] RepositoryClient:52 - version 1.0.273-SNAPSHOT (20160810.120216-3) is not newer than the lastKnownVersion1.0.273-SNAPSHOT (20160810.120216-3)

[bitionaire]

I'm glad that I could finally help. I will clean up the mess sometime soon and will add further tests to it. 😃

[saundersjr]

I can also confirm it's working for me against an internal Nexus.

Thank you.

On 10 Aug 2016 2:31 p.m., "Johann Böhler" notifications@github.com wrote:

I'm glad that I could finally help. I will clean up the mess sometime soon and will add further tests to it. 😃

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/1and1/go-maven-poller/issues/9#issuecomment-238866973, or mute the thread https://github.com/notifications/unsubscribe-auth/AGmNsbY5YEa431Q28v6S_x_vw7B0lcqvks5qedKggaJpZM4JNV_8 .

[jmauntel]

Thanks for putting the time in to fix this issue. It's going to simplify a bunch of workarounds we've been putting in place.

[jmauntel]

I'm not sure if this is related to this bugfix or if it should be a new issue, but when you select a parameratized build, the versions are all showing that they are "about 1 year" old when they are really less than a day. The same thing occurs on release pipelines. Is this related?

[bitionaire]

I have a hard coded timestamp within the code, as I'm only able to extract the build timestamp for the latest build and not for all versions in the list. But I guess that information should be enough, as GoCD seems to store the build information of all latest versions in the internal database. Otherwise you would see only one element per SNAPSHOT version in your parameterized build. I'll try to fix that when I start to clean up the code for this issue. That'll be probably on the weekend. I don't have much time today & tomorrow.

[bitionaire]

Ok, 30 minutes just opened up and I tried to fix the timestamp stuff. You may try 1.1.2-rc1 in order to get the timestamps for the builds. The first build was marked as 2016 years ago for me.. I don't know why, but the timestamp for the next build was ok.

You might experience timezone issues, as I did. Maybe I'll have to add a property to the repo configuration in order to fix this.

[thunder-spb]

Thanks @bitionaire , we'll test this!

[thunder-spb]

@bitionaire yep, fixed now. Thanks!

[bitionaire]

Just uploaded the final version in release 1.1.2 which includes a Time zone property in the repository configuration. E.g. Maven central uses GMT (UTC) for all timestamps. As my GoCD server is located in GMT+2, the latest artifacts were always shown as build 2 hours ago. Once I set the Time zone property to GMT in the repository configuration everything was correct.

If nothing is specified the default will be the system timezone (TimeZone.getDefault()).