search

jenkinsci / azure-vm-agents-plugin

This repo is for azure vm agents plugin for jenkins. Azure devops CICD is the team which owns it for now
https://plugins.jenkins.io/azure-vm-agents/
46 stars 102 forks source link

Unable to access mapped network drives #350

Open ramakrishnaraov opened 2 years ago

ramakrishnaraov commented 2 years ago

Jenkins and plugins versions report

Environment ```text Jenkins: 2.303.3 OS: Linux - 5.4.0-1061-azure ``` --- Parameterized-Remote-Trigger:3.1.5.1 ace-editor:1.1 ant:1.13 antisamy-markup-formatter:2.7 apache-httpcomponents-client-4-api:4.5.13-1.0 artifactory:3.15.4 authentication-tokens:1.4 aws-credentials:189.v3551d5642995 aws-java-sdk-ec2:1.12.148-310.v5e3b_c2681d79 aws-java-sdk-minimal:1.12.148-310.v5e3b_c2681d79 azure-ad:189.v2da14dccdb43 azure-app-service:1.0.2 azure-cli:0.9 azure-commons:1.1.3 azure-container-agents:220.v02fd99997b1a azure-credentials:198.vf9c2fdfde55c azure-keyvault:131.v867845ef6ae9 azure-sdk:84.v53035e83f3c2 azure-vm-agents:810.v0a97a847315a blueocean:1.25.3 blueocean-autofavorite:1.2.5 blueocean-bitbucket-pipeline:1.25.3 blueocean-commons:1.25.3 blueocean-config:1.25.3 blueocean-core-js:1.25.3 blueocean-dashboard:1.25.3 blueocean-display-url:2.4.1 blueocean-events:1.25.3 blueocean-git-pipeline:1.25.3 blueocean-github-pipeline:1.25.3 blueocean-i18n:1.25.3 blueocean-jwt:1.25.3 blueocean-personalization:1.25.3 blueocean-pipeline-api-impl:1.25.3 blueocean-pipeline-editor:1.25.3 blueocean-pipeline-scm-api:1.25.3 blueocean-rest:1.25.3 blueocean-rest-impl:1.25.3 blueocean-web:1.25.3 bootstrap4-api:4.6.0-3 bootstrap5-api:5.1.3-4 bouncycastle-api:2.25 branch-api:2.7.0 build-with-parameters:1.6 caffeine-api:2.9.2-29.v717aac953ff3 checks-api:1.7.2 cloud-stats:0.27 cloudbees-bitbucket-branch-source:757.vddedc5f2589a_ cloudbees-folder:6.17 command-launcher:1.6 common-nimble-plugin:2021.5 config-file-provider:3.8.2 configuration-as-code:1346.ve8cfa_3473c94 copyartifact:1.46.2 credentials:2.6.1 credentials-binding:1.27.1 customizationhooks:3.4.0 dark-theme:0.0.12 display-url-api:2.3.5 docker-commons:1.19 docker-java-api:3.2.13-37.vf3411c9828b9 durable-task:493.v195aefbb0ff2 echarts-api:5.2.2-2 email-ext:2.87 execute-build-toolchain:3.7.1_test_03_alpha extended-read-permission:3.2 favorite:2.3.3 font-awesome-api:5.15.4-5 generic-webhook-trigger:1.83 ghprb:1.42.2 git:4.10.3 git-client:3.11.0 git-server:1.10 github:1.34.1 github-api:1.301-378.v9807bd746da5 github-branch-source:2.11.4 github-checks:1.0.17 github-pr-coverage-status:2.1.1 github-pullrequest:0.4.0 github-scm-trait-commit-skip:0.4.0 gradle:1.38 handlebars:3.0.8 handy-uri-templates-2-api:2.1.8-1.0 htmlpublisher:1.28 http_request:1.14 ivy:2.1 jackson2-api:2.13.1-246.va8a9f3eaf46a javadoc:217.v905b_86277a_2a_ javax-activation-api:1.2.0-2 javax-mail-api:1.6.2-5 jaxb:2.3.0.1 jdk-tool:1.5 jenkins-design-language:1.25.3 jjwt-api:0.11.2-9.c8b45b8bb173 jquery-detached:1.2.1 jquery3-api:3.6.0-2 jsch:0.1.55.2 junit:1.53 kubernetes:1.30.11 kubernetes-client-api:5.11.2-182.v0f1cf4c5904e kubernetes-credentials:0.9.0 lockable-resources:2.13 logstash:2.5.0205.vd05825ed46bd mailer:408.vd726a_1130320 matrix-auth:3.0.1 matrix-project:1.20 maven-plugin:3.16 metrics:4.0.2.8.1 momentjs:1.1.1 okhttp-api:4.9.3-105.vb96869f8ac3a periodicbackup:1.8 pipeline-build-step:2.15 pipeline-github:2.8-138.d766e30bb08b pipeline-github-lib:36.v4c01db_ca_ed16 pipeline-githubnotify-step:1.0.5 pipeline-graph-analysis:1.11 pipeline-input-step:446.vf27b_0b_83500e pipeline-milestone-step:1.3.2 pipeline-model-api:1.9.3 pipeline-model-definition:1.9.3 pipeline-model-extensions:1.9.3 pipeline-rest-api:2.21 pipeline-stage-step:291.vf0a8a7aeeb50 pipeline-stage-tags-metadata:1.9.3 pipeline-stage-view:2.21 pipeline-utility-steps:2.12.0 plain-credentials:1.8 plugin-util-api:2.13.0 popper-api:1.16.1-2 popper2-api:2.11.2-1 pubsub-light:1.16 rebuild:1.33 rtc-client-plain-api:6.0.3 scm-api:595.vd5a_df5eb_0e39 script-security:1131.v8b_b_5eda_c328e snakeyaml-api:1.30.1 sse-gateway:1.25 ssh-agent:1.24.1 ssh-credentials:1.19 ssh-slaves:1.33.0 sshd:3.1.0 strict-crumb-issuer:2.1.0 structs:308.v852b473a2b8c teamconcert:2.4.1 teamconcert-git:2.0.3 theme-manager:0.6 timestamper:1.17 token-macro:267.vcdaea6462991 trilead-api:1.0.13 update-sites-manager:2.0.0 update-toolbase:2021.5 variant:1.4 windows-azure-storage:365.vf41653c43b01 workflow-aggregator:2.6 workflow-api:1136.v7f5f1759dc16 workflow-basic-steps:2.24 workflow-cps:2648.va9433432b33c workflow-cps-global-lib:552.vd9cc05b8a2e1 workflow-durable-task-step:2.40 workflow-job:1145.v7f2433caa07f workflow-multibranch:706.vd43c65dec013 workflow-scm-step:2.13 workflow-step-api:622.vb_8e7c15b_c95a_ workflow-support:813.vb_d7c3d2984a_0

What Operating System are you using (both controller, and any agents involved in the problem)?

Jenkins is hosted in AKS Windows based VM agents

Reproduction steps

  1. Create Azure File Share in storage account and upload some files
  2. Create a Virtual Machine with Windows 10
  3. Connect the File share in VM as a mounted drive (using storage account)
  4. Create an image out of the Virtual Machine
  5. Create dynamic slave from the above image using Azure VM Agents plugin
  6. Login to the VM check if the mapped drive is available with contents
  7. Create a Pipeline Job to dir the contents of mapped drive

Expected Results

The contents of the mapped network drive should be displayed.

Actual Results

File Not found - error message

Anything else?

No response

timja commented 2 years ago

How are you mounting the drive? Can you give a bit more details please. The steps above aren't enough given I'm not too familiar with that area

ramakrishnaraov commented 2 years ago

Dear @timja, Sure, please find further details.

Azure File Shares can be connected to VMs as mapped network drive. https://docs.microsoft.com/en-us/azure/storage/files/storage-how-to-use-files-windows#mount-the-azure-file-share New-PSDrive -Name -PSProvider FileSystem -Root -Persist

The drive is created and accessible when we login to the VM. But it is not accessible (from Jenkins Pipeline) in the way how the slaves are connected to Jenkins. I was using SSH lunching method for connection of slaves to Jenkins.

When I manually create node (slave agent in Jenkins) and connect the VM via JNLP in command prompt, I am able to access the mapped drive from Jenkins Pipeline. (Note: I have the enable the use WebSocket) option in Jenkins slave agent configuration to establish the connection via JNLP) image

But I am not able to use JNLP lunch option for agents created via Azure VM Agents plugin as it does not have a provision to enable usage of WebSocket.

I have enabled fixed port (5000) for JNLP connection, but this port is not reachable from Jenkins slaves.

ramakrishnaraov commented 2 years ago

Dear @timja, I am able to connect the dynamic VM agents via JNLP (using websocket) with the following command in the initialization script: $java -cp C:\agent.jar hudson.remoting.jnlp.Main -url "$jenkinsserverurl" -webSocket -workDir "C:\workspace\" $secret "$vmname"

But still the mapped drive is not visible from the Jenkins Pipeline. Any hints on how to access the file share from Jenkins VM agents would be greatly helpful.

Thanks, Ram

timja commented 2 years ago

is there a bat or powershell script you can run to mount the drive?

ramakrishnaraov commented 2 years ago

Yes, there is a powershell script to mount the drive. I tried doing it directly from Jenkins Pipeline job, the drive is getting created but it is not accessible.

Pipeline script: // The following powershell script successfully creates a mount drive (T:) for the Azure file share (test-fs): powershell ''' $pass="$env:storage_key"|ConvertTo-SecureString -AsPlainText -Force $Cred = New-Object System.Management.Automation.PsCredential('localhost\labXXXst',$pass) New-PSDrive -Name T -PSProvider FileSystem -Root "\\labXXXst.file.core.windows.net\test-fs" -Credential $cred -Persist ''' // Linking happens successfully -> mapped drive and contents exists bat 'mklink /d C:\tb_pub_point T:\tbpubpoint1\data' // but they are not accessible via Jenkins bat 'dir C:\tb_pub_point'

Output: [Pipeline] powershell

Name Used (GB) Free (GB) Provider Root CurrentLocation

T 1816.06 3303.94 FileSystem \labXXXst.file.core.windows....

[Pipeline] bat

cibuilduser@CX-CLOUD-VM c:\jenkins\workspace\workspace\vaa3cob\TB_Update>mklink /d C:\tb_pub_point T:\tbpubpoint1\data symbolic link created for C:\tb_pub_point <<===>> T:\tbpubpoint1\data [Pipeline] bat

cibuilduser@CX-CLOUD-VM c:\jenkins\workspace\workspace\vaa3cob\TB_Update>dir C:\tb_pub_point Volume in drive C is Windows Volume Serial Number is CC5D-425B

Directory of C:\tb_pub_point

File Not Found