search

jenkinsci / azure-storage-plugin

21 stars 45 forks source link

Cannot Upload Empty Files to File share #210

Closed elkh510 closed 2 years ago

elkh510 commented 2 years ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.319.1 OS: Linux - 5.4.120+ --- ace-editor:1.1 allure-jenkins:3.26.1 allure-jenkins-plugin:2.29.0 ansicolor:1.0.1 ant:1.13 antisamy-markup-formatter:2.5 apache-httpcomponents-client-4-api:4.5.13-1.0 artifactory:3.14.2 authentication-tokens:1.4 authorize-project:1.4.0 autocomplete-parameter:1.1 azure-ad:188.v2369adb95a31 azure-artifact-manager:97.v074e1332e88d azure-commons:1.1.3 azure-credentials:198.vf9c2fdfde55c azure-sdk:70.v63f6a95999a7 basic-branch-build-strategies:1.3.2 blackduck-detect:7.0.0 blueocean:1.25.2 blueocean-autofavorite:1.2.4 blueocean-bitbucket-pipeline:1.25.2 blueocean-commons:1.25.2 blueocean-config:1.25.2 blueocean-core-js:1.25.2 blueocean-dashboard:1.25.2 blueocean-display-url:2.4.1 blueocean-events:1.25.2 blueocean-git-pipeline:1.25.2 blueocean-github-pipeline:1.25.2 blueocean-i18n:1.25.2 blueocean-jira:1.25.2 blueocean-jwt:1.25.2 blueocean-personalization:1.25.2 blueocean-pipeline-api-impl:1.25.2 blueocean-pipeline-editor:1.25.2 blueocean-pipeline-scm-api:1.25.2 blueocean-rest:1.25.2 blueocean-rest-impl:1.25.2 blueocean-web:1.25.2 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.3-3 bouncycastle-api:2.25 branch-api:2.7.0 build-monitor-plugin:1.13+build.202112061615 build-user-vars-plugin:1.8 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloudbees-bitbucket-branch-source:734.v2f848c5e6ea2 cloudbees-disk-usage-simple:0.10 cloudbees-folder:6.16 cobertura:1.17 code-coverage-api:2.0.4 command-launcher:1.6 config-file-provider:3.8.2 copyartifact:1.46.2 credentials:1055.v1346ba467ba1 credentials-binding:1.27 data-tables-api:1.11.3-4 display-url-api:2.3.5 docker-commons:1.17 docker-workflow:1.26 dtkit-api:3.0.0 durable-task:493.v195aefbb0ff2 echarts-api:5.2.2-1 extended-choice-parameter:0.82 extended-read-permission:3.2 external-monitor-job:1.7 favorite:2.3.3 file-operations:1.11 font-awesome-api:5.15.4-4 forensics-api:1.7.0 generic-webhook-trigger:1.77 git:4.10.1 git-changelog:3.14 git-client:3.10.0 git-parameter:0.9.14 git-server:1.10 github:1.34.1 github-api:1.301-378.v9807bd746da5 github-autostatus:3.6.2 github-branch-source:2.11.3 github-oauth:0.36 github-pullrequest:0.3.0 google-oauth-plugin:1.0.6 gradle:1.37.1 greenballs:1.15.1 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-1.0 htmlpublisher:1.28 http_request:1.12 icon-shim:3.0.0 ivy:2.1 jackson2-api:2.13.0-230.v59243c64b0a5 jacoco:3.3.0 javadoc:1.6 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.2 jira:3.6 jjwt-api:0.11.2-9.c8b45b8bb173 job-dsl:1.78.1 jquery:1.12.4-1 jquery-detached:1.2.1 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 kubernetes:1.31.1 kubernetes-client-api:5.10.1-171.vaa0774fb8c20 kubernetes-credentials:0.9.0 ldap:2.7 locale:1.4 lockable-resources:2.12 logstash:2.5.0205.vd05825ed46bd mailer:1.34 matrix-auth:3.0 matrix-project:1.19 maven-plugin:3.15.1 mercurial:2.16 metrics:4.0.2.8 momentjs:1.1.1 multibranch-action-triggers:1.8.6 multibranch-build-strategy-extension:1.0.10 next-build-number:1.7 oauth-credentials:0.5 okhttp-api:4.9.3-105.vb96869f8ac3a pam-auth:1.6.1 parameterized-scheduler:1.0 pipeline-build-step:2.15 pipeline-github:2.8-138.d766e30bb08b pipeline-graph-analysis:1.12 pipeline-input-step:427.va6441fa17010 pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.3 pipeline-model-declarative-agent:1.1.1 pipeline-model-definition:1.9.3 pipeline-model-extensions:1.9.3 pipeline-rest-api:2.19 pipeline-stage-step:2.5 pipeline-stage-tags-metadata:1.9.3 pipeline-stage-view:2.19 pipeline-utility-steps:2.11.0 plain-credentials:1.7 plugin-util-api:2.7.0 popper-api:1.16.1-2 popper2-api:2.10.2-1 pubsub-light:1.16 run-condition:1.5 saml:2.0.9 scm-api:2.6.5 script-security:1118.vba21ca2e3286 slack:2.49 snakeyaml-api:1.29.1 sonar:2.14 sse-gateway:1.24 ssh-credentials:1.19 sshd:3.1.0 structs:308.v852b473a2b8c timestamper:1.15 token-macro:267.vcdaea6462991 trilead-api:1.0.13 variant:1.4 windows-azure-storage:365.vf41653c43b01 windows-slaves:1.8 workflow-aggregator:2.6 workflow-api:1108.v57edf648f5d4 workflow-basic-steps:2.24 workflow-cps:2648.va9433432b33c workflow-cps-global-lib:552.vd9cc05b8a2e1 workflow-durable-task-step:1107.v5dab75aaccbd workflow-job:1145.v7f2433caa07f workflow-multibranch:2.26 workflow-scm-step:2.13 workflow-step-api:615.vb09dac339255 workflow-support:804.vba10a18a1476 xunit:3.0.5 ```

What Operating System are you using (both controller, and any agents involved in the problem)?

docker images: jenkins - 2.319.1-lts-jdk11 agent - 4.11-1-jdk11

Reproduction steps

run pipeline

node('master') {
    deleteDir()
    script{
        String randomString = org.apache.commons.lang.RandomStringUtils.random(9, true, true)
        def file = "kk${randomString}k.txt"
        sh "touch ${file}"

        azureUpload fileShareName: 'fileShareName',
                    filesPath: file,
                    virtualPath: "justForTestCanBeDeleted",
                    flattenDirectories: false,
                    storageCredentialId: 'storageCredentialId',
                    storageType: 'filestorage',
                    verbose: true
    }
}

Expected Results

successful upload

Actual Results


AzureStorage - Container name: 
AzureStorage - share name: fileShareName
AzureStorage - File path: kkaFB88BzPnk.txt
AzureStorage - Virtual path: justForTestCanBeDeleted/
AzureStorage - Exclude path: 
AzureStorage - Exclude prefix: 
AzureStorage - Uploading files to Azure
AzureStorage - Need uploaded/archived file count = 1
ERROR: AzureStorage - Error occurred while uploading to Azure - jenkinsng
com.microsoftopentechnologies.windowsazurestorage.exceptions.WAStorageException: Only 0/1 files are successfully uploaded.
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadService.execute(UploadService.java:563)
    at com.microsoftopentechnologies.windowsazurestorage.WAStoragePublisher.perform(WAStoragePublisher.java:472)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
com.microsoftopentechnologies.windowsazurestorage.exceptions.WAStorageException: Only 0/1 files are successfully uploaded.
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadService.execute(UploadService.java:563)
    at com.microsoftopentechnologies.windowsazurestorage.WAStoragePublisher.perform(WAStoragePublisher.java:472)
Caused: java.io.IOException: AzureStorage - Error occurred while uploading to Azure - jenkinsng
    at com.microsoftopentechnologies.windowsazurestorage.WAStoragePublisher.perform(WAStoragePublisher.java:502)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
Finished: FAILURE

Anything else?

No response

elopsod commented 2 years ago

hi have same issue any ETA on this ?

LizWaitAlt commented 2 years ago

hello!

I have the same issue as well. Looks like there have been multiple bugs reported with a similar issue. Would appreciate an update :) Thank you in advance!

timja commented 2 years ago

It’s planned but I haven’t had time yet, PRs are welcome though

timja commented 2 years ago

Reproduced, the relevant error is in the system log:

2022-02-03 09:00:23.374+0000 [id=129]   SEVERE  c.a.c.util.logging.ClientLogger#performLogging: java.lang.IllegalArgumentException: ShareFileRange end must be greater than or equal to 0 if specified.
2022-02-03 09:00:23.374+0000 [id=129]   SEVERE  c.m.w.s.UploadToFileService$UploadOnAgent#uploadCloudFile: Failed uploading file
java.lang.IllegalArgumentException: ShareFileRange end must be greater than or equal to 0 if specified.
Also:   java.lang.Exception: #block terminated with an error
        at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:99)
        at reactor.core.publisher.Mono.block(Mono.java:1706)
Caused: java.lang.IllegalArgumentException
    at com.azure.storage.file.share.models.ShareFileRange.<init>(ShareFileRange.java:43)
    at com.azure.storage.file.share.ShareFileAsyncClient.uploadRangeWithResponse(ShareFileAsyncClient.java:1644)
    at com.azure.storage.file.share.ShareFileAsyncClient.lambda$uploadWithResponse$33(ShareFileAsyncClient.java:1508)
    at com.azure.storage.common.implementation.UploadUtils.lambda$uploadFullOrChunked$0(UploadUtils.java:73)
    at reactor.core.publisher.FluxSwitchOnFirst$AbstractSwitchOnFirstMain.onComplete(FluxSwitchOnFirst.java:557)
    at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onComplete(FluxConcatMapNoPrefetch.java:240)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onComplete(FluxFilterFuseable.java:171)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onComplete(FluxFilterFuseable.java:391)
    at reactor.core.publisher.FluxGenerate$GenerateSubscription.complete(FluxGenerate.java:196)
    at com.azure.core.util.FluxUtil.lambda$toFluxByteBuffer$2(FluxUtil.java:222)
    at reactor.core.publisher.FluxGenerate$GenerateSubscription.slowPath(FluxGenerate.java:265)
    at reactor.core.publisher.FluxGenerate$GenerateSubscription.request(FluxGenerate.java:207)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.request(FluxFilterFuseable.java:411)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.request(FluxFilterFuseable.java:191)
    at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.request(FluxConcatMapNoPrefetch.java:336)
    at reactor.core.publisher.FluxSwitchOnFirst$AbstractSwitchOnFirstMain.onSubscribe(FluxSwitchOnFirst.java:437)
    at reactor.core.publisher.FluxConcatMapNoPrefetch$FluxConcatMapNoPrefetchSubscriber.onSubscribe(FluxConcatMapNoPrefetch.java:164)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableSubscriber.onSubscribe(FluxFilterFuseable.java:87)
    at reactor.core.publisher.FluxFilterFuseable$FilterFuseableConditionalSubscriber.onSubscribe(FluxFilterFuseable.java:305)
    at reactor.core.publisher.FluxGenerate.subscribe(FluxGenerate.java:84)
    at reactor.core.publisher.Mono.subscribe(Mono.java:4361)
    at reactor.core.publisher.Mono.block(Mono.java:1705)
    at com.azure.storage.common.implementation.StorageImplUtils.blockWithOptionalTimeout(StorageImplUtils.java:128)
    at com.azure.storage.file.share.ShareFileClient.uploadWithResponse(ShareFileClient.java:1104)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService$UploadOnAgent.uploadCloudFile(UploadToFileService.java:165)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService$UploadOnAgent.lambda$invoke$0(UploadToFileService.java:148)
    at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
    at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
    at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
    at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
    at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:952)
    at java.base/java.util.stream.ReduceOps$ReduceTask.doLeaf(ReduceOps.java:926)
    at java.base/java.util.stream.AbstractTask.compute(AbstractTask.java:327)
    at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:746)
    at java.base/java.util.concurrent.ForkJoinTask.doExec$$$capture(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java)
    at java.base/java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:408)
    at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:736)
    at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateParallel(ReduceOps.java:919)
    at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
    at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService$UploadOnAgent.invoke(UploadToFileService.java:151)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService$UploadOnAgent.invoke(UploadToFileService.java:119)
    at hudson.FilePath.act(FilePath.java:1170)
    at hudson.FilePath.act(FilePath.java:1153)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService.uploadIndividuals(UploadToFileService.java:100)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadToFileService.uploadIndividuals(UploadToFileService.java:72)
    at com.microsoftopentechnologies.windowsazurestorage.service.UploadService.execute(UploadService.java:545)
    at com.microsoftopentechnologies.windowsazurestorage.WAStoragePublisher.perform(WAStoragePublisher.java:472)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:100)
    at org.jenkinsci.plugins.workflow.steps.CoreStep$Execution.run(CoreStep.java:70)
    at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:264)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:829)
timja commented 2 years ago

Reported upstream to https://github.com/Azure/azure-sdk-for-java/issues/26867

I wasn't able to figure out a way to get this working, self contained reproducer at https://github.com/timja/azure-empty-file-reproducer/blob/master/src/test/java/UploadEmptyFileTest.java if anyone wants to play with it outside of Jenkins