jfrog / artifactory-client-java

Artifactory REST Client Java API bindings
Apache License 2.0
319 stars 153 forks source link

Fails to upload artifact greater than 4GB in size #367

Closed sactiw13 closed 1 year ago

sactiw13 commented 1 year ago

Fails to upload artifact greater than 4GB in size while upload with 'curl' passes

Steps 1) create/fetch a file (artifact) of size greater than 4GB. 2) try uploading it to generic repo in artifactory with using artifactory-client-java 3) it should fails with following error:

groovyx.net.http.HttpResponseException: status code: 409, reason phrase: {
  "errors" : [ {
    "status" : 409,
    "message" : "Checksum policy 'LocalRepoChecksumPolicy: CLIENT' rejected the artifact 'xxx'. Checksums info: ChecksumsInfo{checksums={SHA-1=ChecksumInfo{type=SHA-1, original='1bf99ee9f374e58e201e4dda4f474e570eb77229', actual='da39a3ee5e6b4b0d3255bfef95601890afd80709'}, MD5=ChecksumInfo{type=MD5, original='null', actual='d41d8cd98f00b204e9800998ecf8427e'}, SHA-256=ChecksumInfo{type=SHA-256, original='null', actual='e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}}}"
  } ]
}

Expected behavior It should not fail (note: I have tried with file size 1GB, 2GB, 3GB for them this utility works)

Screenshots Below log excerpts shows that it passes for 1GB, 2GB, and 3GB but fails for 4GB file

2023-02-21 22:53:24,636 INFO  Builder_Service - Uploading file /xxx/p4/build/sw/1gb.file
2023-02-21 22:53:25.259  INFO 483287 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Starting Artifactory-client 2.13.0 using Java 1.8.0_201-ojdkbuild on xxx-ub20-vb-3 with PID 483287
2023-02-21 22:53:25.956  INFO 483287 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT BEGINS-----------------
2023-02-21 22:54:10.614  INFO 483287 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Path where the artifact has been uploaded is: https://urm.xxx.com/artifactory/xxx-internal-generic-local/2.0-dummy1/1gb.file
2023-02-21 22:54:10.614  INFO 483287 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT FINISHED SUCCESSFULLY-----------------
2023-02-21 22:54:10,636 INFO  Builder_Service - Artifactory utility 'upload' command executed successfully !!

2023-02-21 22:54:10,636 INFO  Builder_Service - Uploading file /xxx/p4/build/sw/2gb.file
2023-02-21 22:54:11.260  INFO 483362 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Starting Artifactory-client 2.13.0 using Java 1.8.0_201-ojdkbuild on xxx-ub20-vb-3 with PID 483362
2023-02-21 22:54:11.956  INFO 483362 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT BEGINS-----------------
2023-02-21 22:54:34.333  INFO 483362 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Path where the artifact has been uploaded is: https://urm.xxx.com/artifactory/xxx-internal-generic-local/2.0-dummy1/2gb.file
2023-02-21 22:54:34.333  INFO 483362 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT FINISHED SUCCESSFULLY-----------------
2023-02-21 22:54:34,353 INFO  Builder_Service - Artifactory utility 'upload' command executed successfully !!

2023-02-21 22:54:34,354 INFO  Builder_Service - Uploading file /xxx/p4/build/sw/3gb.file
2023-02-21 22:54:34.986  INFO 483421 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Starting Artifactory-client 2.13.0 using Java 1.8.0_201-ojdkbuild on xxx-ub20-vb-3 with PID 483421
2023-02-21 22:54:35.675  INFO 483421 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT BEGINS-----------------
2023-02-21 22:55:42.003  INFO 483421 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Path where the artifact has been uploaded is: https://urm.xxx.com/artifactory/xxx-internal-generic-local/2.0-dummy1/3gb.file
2023-02-21 22:55:42.003  INFO 483421 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT FINISHED SUCCESSFULLY-----------------
2023-02-21 22:55:42,032 INFO  Builder_Service - Artifactory utility 'upload' command executed successfully !!

2023-02-21 22:55:42,033 INFO  Builder_Service - Uploading file /xxx/p4/build/sw/4gb.file

2023-02-21 22:55:42.684  INFO 483519 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Starting Artifactory-client 2.13.0 using Java 1.8.0_201-ojdkbuild on xxx-ub20-vb-3 with PID 483519
2023-02-21 22:55:43.414  INFO 483519 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : -------------UPLOADING ARTIFACT BEGINS-----------------
2023-02-21 22:56:00.870 ERROR 483519 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : Uploading artifact failed due to groovyx.net.http.HttpResponseException: status code: 409, reason phrase: {
  "errors" : [ {
    "status" : 409,
    "message" : "Checksum policy 'LocalRepoChecksumPolicy: CLIENT' rejected the artifact 'xxx-internal-generic-local:2.0-dummy1/4gb.file'. Checksums info: ChecksumsInfo{checksums={SHA-1=ChecksumInfo{type=SHA-1, original='1bf99ee9f374e58e201e4dda4f474e570eb77229', actual='da39a3ee5e6b4b0d3255bfef95601890afd80709'}, MD5=ChecksumInfo{type=MD5, original='null', actual='d41d8cd98f00b204e9800998ecf8427e'}, SHA-256=ChecksumInfo{type=SHA-256, original='null', actual='e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855'}}}"
  } ]
}
2023-02-21 22:56:11.961  INFO 483519 --- [           main] c.n.i.a.ArtifactoryUtilitiesApplication  : --------------COULDN'T UPLOAD THE ARTIFACT-----------------

Versions

Additional context NOTE: the upload fails within 1 min of start but when I try to upload same '>4GB' file using 'curl' then upload passes without any failure

sactiw13 commented 1 year ago

We have root caused the issue, and it seems that artifactory-utilities wrapper jar deployed in production instance was build with 2.6.0 version of artifactory java client.

So it seems we missed updating the artifactory-utilities wrapper jar after updating the Artifactory Java Client version from 2.6.0 to 2.13.0

## 2.8.5 (September 12, 2019)
- Fix failure to upload large files.

So, all in all it was deployment issue from start as we had missed updating the artifactory-utilities wrapper jar after Artifactory Java Client version upgrade from 2.6.0 to 2.13.0

Sorry for the confusion, closing the ticket as false alarm.