microsoft / azure-pipelines-agent

Azure Pipelines Agent 🚀
MIT License
1.73k stars 869 forks source link

[Question]: Can we configure number of repo clones in self-hosted agent #4624

Open nishanthpasham opened 10 months ago

nishanthpasham commented 10 months ago

Describe your question

We're running self-hosted Azure agents for our CIs. Our codebase is big ~30GB. By default, looks like the agent can clone the code up to 6 times, while running different jobs. When this happens, we are running out of disk space on our CI VM.

Ideally we'd like to configure the max number of code clones when setting up the self-hosted agent. Is there a way to specify this?

image

Versions

agent v3.232.1

Environment type (Please select at least one enviroment where you face this issue)

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Operation system

Rocky Linux 9.3

Version controll system

git

Azure DevOps Server Version (if applicable)

No response

aleksandrlevochkin commented 10 months ago

Hi @nishanthpasham, thank you for your question. You can use the pipeline variable AZP_AGENT_ALLOW_WORK_DIRECTORY_REPOSITORIES to clone the code to a common location in the work directory.

Example - test1.yml:

trigger:
- main

pool: 'Default'

variables:
  AZP_AGENT_ALLOW_WORK_DIRECTORY_REPOSITORIES: "true"

jobs:
- job: Test1
  steps:
  - checkout: self
    path: ../repos/test
  - script: echo Test2

Example - test2.yml:

trigger:
- main

pool: 'Default'

variables:
  AZP_AGENT_ALLOW_WORK_DIRECTORY_REPOSITORIES: "true"

jobs:
- job: Test2
  steps:
  - checkout: self
    path: ../repos/test
  - script: echo Test2

In this case, the repository for both build definitions will be placed in the same directory (_work\repos\test), not taking up additional disk space.

nishanthpasham commented 9 months ago

Tried this. But it does not seem to be working. Please suggest @aleksandrlevochkin image

variables are defined in the pipeline as below image

aleksandrlevochkin commented 7 months ago

@nishanthpasham Hi, apologies for the late response. Is this issue still relevant for you? The way you added the variable seems correct. What version of the agent are you using? This knob is available from v3.230.0 onwards.

If the issue is not related to the version, could you please check if the knob is present in the agent logs (and if it is set to true there)?

kevinjc31 commented 2 months ago

I have set AZP_AGENT_ALLOW_WORK_DIRECTORY_REPOSITORIES=true and I'm able to and my checkout is working. However all my Bash@3 scripts are now broken with the following error (Error: spawn /__a/externals/node20_1/bin/node ENOENT)