jenkinsci / azure-storage-plugin

21 stars 45 forks source link

Cannot download files from BLOB via jenkins #219

Closed elkh510 closed 2 years ago

elkh510 commented 2 years ago

Jenkins and plugins versions report

Environment Jenkins: 2.319.1 OS: Linux - 5.4.120+ --- ace-editor:1.1 allure-jenkins:3.26.1 allure-jenkins-plugin:2.29.0 ansicolor:1.0.1 ant:1.13 antisamy-markup-formatter:2.5 apache-httpcomponents-client-4-api:4.5.13-1.0 artifactory:3.14.2 authentication-tokens:1.4 authorize-project:1.4.0 autocomplete-parameter:1.1 azure-ad:185.v3b416408dcb1 azure-artifact-manager:97.v074e1332e88d azure-commons:1.1.3 azure-credentials:198.vf9c2fdfde55c azure-sdk:85.v4817a_b_8a_7124 basic-branch-build-strategies:1.3.2 blackduck-detect:7.0.0 blueocean:1.25.2 blueocean-autofavorite:1.2.4 blueocean-bitbucket-pipeline:1.25.2 blueocean-commons:1.25.2 blueocean-config:1.25.2 blueocean-core-js:1.25.2 blueocean-dashboard:1.25.2 blueocean-display-url:2.4.1 blueocean-events:1.25.2 blueocean-git-pipeline:1.25.2 blueocean-github-pipeline:1.25.2 blueocean-i18n:1.25.2 blueocean-jira:1.25.2 blueocean-jwt:1.25.2 blueocean-personalization:1.25.2 blueocean-pipeline-api-impl:1.25.2 blueocean-pipeline-editor:1.25.2 blueocean-pipeline-scm-api:1.25.2 blueocean-rest:1.25.2 blueocean-rest-impl:1.25.2 blueocean-web:1.25.2 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.3-3 bouncycastle-api:2.25 branch-api:2.7.0 build-monitor-plugin:1.13+build.202112061615 build-user-vars-plugin:1.8 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloudbees-bitbucket-branch-source:734.v2f848c5e6ea2 cloudbees-disk-usage-simple:0.10 cloudbees-folder:6.16 cobertura:1.17 code-coverage-api:2.0.4 command-launcher:1.6 config-file-provider:3.8.2 configuration-as-code:1.55 copyartifact:1.46.2 credentials:2.6.2 credentials-binding:1.27 data-tables-api:1.11.3-4 display-url-api:2.3.5 docker-commons:1.17 docker-workflow:1.26 dtkit-api:3.0.0 durable-task:493.v195aefbb0ff2 echarts-api:5.2.2-1 extended-choice-parameter:0.82 extended-read-permission:3.2 external-monitor-job:1.7 favorite:2.3.3 file-operations:1.11 font-awesome-api:5.15.4-4 forensics-api:1.7.0 generic-webhook-trigger:1.77 git:4.10.0 git-changelog:3.14 git-client:3.10.0 git-parameter:0.9.13 git-server:1.10 github:1.34.1 github-api:1.301-378.v9807bd746da5 github-autostatus:3.6.2 github-branch-source:2.11.3 github-oauth:0.35 github-pullrequest:0.3.0 google-oauth-plugin:1.0.6 gradle:1.37.1 greenballs:1.15.1 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-1.0 htmlpublisher:1.28 http_request:1.12 icon-shim:3.0.0 ivy:2.1 jackson2-api:2.13.0-230.v59243c64b0a5 jacoco:3.3.0 javadoc:1.6 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.2 jira:3.6 jjwt-api:0.11.2-9.c8b45b8bb173 job-dsl:1.78.1 jquery:1.12.4-1 jquery-detached:1.2.1 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 kubernetes:1.31.0 kubernetes-client-api:5.10.1-171.vaa0774fb8c20 kubernetes-credentials:0.9.0 ldap:2.7 locale:1.4 lockable-resources:2.12 logstash:2.5.0205.vd05825ed46bd mailer:1.34 matrix-auth:2.6.11 matrix-project:1.19 maven-plugin:3.15.1 mercurial:2.16 metrics:4.0.2.8 momentjs:1.1.1 multibranch-action-triggers:1.8.6 multibranch-build-strategy-extension:1.0.10 next-build-number:1.7 oauth-credentials:0.5 okhttp-api:4.9.3-105.vb96869f8ac3a pam-auth:1.6.1 parameterized-scheduler:1.0 pipeline-build-step:2.15 pipeline-github:2.8-138.d766e30bb08b pipeline-graph-analysis:1.12 pipeline-input-step:427.va6441fa17010 pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.3 pipeline-model-declarative-agent:1.1.1 pipeline-model-definition:1.9.3 pipeline-model-extensions:1.9.3 pipeline-rest-api:2.19 pipeline-stage-step:2.5 pipeline-stage-tags-metadata:1.9.3 pipeline-stage-view:2.19 pipeline-utility-steps:2.11.0 plain-credentials:1.7 plugin-util-api:2.6.0 popper-api:1.16.1-2 popper2-api:2.10.2-1 pubsub-light:1.16 run-condition:1.5 saml:2.0.9 scm-api:2.6.5 script-security:1.78 slack:2.49 snakeyaml-api:1.29.1 sonar:2.14 sse-gateway:1.24 ssh-credentials:1.19 sshd:3.1.0 structs:308.v852b473a2b8c timestamper:1.15 token-macro:267.vcdaea6462991 trilead-api:1.0.13 variant:1.4 windows-azure-storage:368.v7443dd3deffe windows-slaves:1.8 workflow-aggregator:2.6 workflow-api:1105.v3de5e2efac97 workflow-basic-steps:2.24 workflow-cps:2640.v00e79c8113de workflow-cps-global-lib:552.vd9cc05b8a2e1 workflow-durable-task-step:1102.v9c8d2f466adb workflow-job:2.42 workflow-multibranch:2.26 workflow-scm-step:2.13 workflow-step-api:613.v375732a042b1 workflow-support:3.8 xunit:3.0.5

What Operating System are you using (both controller, and any agents involved in the problem)?

docker images: jenkins - 2.319.1-lts-jdk11 agent - 4.11-1-jdk11

Reproduction steps

node('master') {
    deleteDir()
    script{
        String randomString1 = org.apache.commons.lang.RandomStringUtils.random(9, true, true)
        String randomString2 = org.apache.commons.lang.RandomStringUtils.random(9, true, true)
        sh "echo '111' > t${randomString1}.yml; echo '222' > t${randomString2}.yml"
        sh "ls -la ."
        azureUpload fileShareName: "justtests",
                    filesPath: '**/t*.yml',
                    virtualPath: "tests",
                    storageCredentialId: 'storageCredentialId',
                    storageType: 'filestorage',
                    verbose: true

        String randomString = org.apache.commons.lang.RandomStringUtils.random(9, true, true)
        def j = "kk${randomString}k.txt"
        sh """
            dd if=/dev/urandom of=${j} bs=1MB count=768
            du -sh ${j} # 240mb
        """
        azureUpload containerName: 'backup', filesPath: j, storageCredentialId: "storageCredentialId", storageType: 'blobstorage'
    }
}

Expected Results

open pipeline(code above) run(or build) click on "Azure Artifacts" click on uploaded file and download it

Actual Results

open pipeline(code above) run(or build) click on "Azure Artifacts" click on uploaded file and get error image

Anything else?

most likely related to the ticket - https://github.com/jenkinsci/azure-storage-plugin/issues/214 since it became reproducible, after updating the version in which the bug from the ticket was fixed

timja commented 2 years ago

Don’t think it can be related that only touched file share code

elopsod commented 2 years ago

hi have same issue @timja do you have any ETA on this please

timja commented 2 years ago

no this is a community maintained project, if you provide a fix it will get there quicker otherwise I'll look when I have time

(wondering if this is related to an Azure SDK upgrade instead as it'd be quite weird if the suggest issue was the cause)

elopsod commented 2 years ago

oh, sorry thank you @timja

timja commented 2 years ago

Reproduced, For the record didn't need a huge file for it. 😄

timja commented 2 years ago

Interestingly it only happens with both files and blobs in the same pipeline by the looks of it

timja commented 2 years ago

Will be fixed in https://github.com/jenkinsci/azure-storage-plugin/pull/220

Was quite a weird one, there was a few bugs in that.

Seems to manifest when you have both files and blobs uploaded in the same run, I haven't tested the older version but I really don't think it was caused by the above issue. Likely just noticed after it.

the fix will only work for newly uploaded files, (you could modify files on disk if you really want to fix them)

elopsod commented 2 years ago

@timja many thanks!)