worldspawn
commented
4 years ago So I apparently needed to be more selective with my container and needed a docker enabled container.
Closed worldspawn closed 4 years ago
worldspawn
commented
4 years ago So I apparently needed to be more selective with my container and needed a docker enabled container.
christopherdalton
commented
4 years ago I have the exact same issue.. what do you mean by a docker enabled container?? How did you fix this?
enriko-riba
commented
3 years ago anyone solved this?
worldspawn
commented
3 years ago Hello, what I meant by a docker enabled container is a container with docker installed. If its not installed it ain't going to run :)
Caveat, if your running this on kubernetes > 1.19.x (i think) you can longer run docker in docker
worldspawn
commented
3 years ago Heres my agent docker file btw
FROM ubuntu:18.04
ENV DEBIAN_FRONTEND=noninteractive
RUN echo "APT::Get::Assume-Yes \"true\";" > /etc/apt/apt.conf.d/90assumeyes
RUN apt-get update
RUN apt-get install -y --no-install-recommends apt-transport-https ca-certificates curl software-properties-common jq git iputils-ping libcurl4 libicu60 libunwind8 netcat libssl1.0 gnupg-agent wget lsb-release gnupg
RUN curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add -
RUN add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu bionic stable" && apt-get update
RUN apt-get install -y --no-install-recommends openssh-client
RUN curl -sL https://aka.ms/InstallAzureCLIDeb | bash
RUN wget -q https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb
RUN apt-get install ./packages-microsoft-prod.deb -y && apt-get update && apt-get install powershell -y
RUN curl -fsSL https://deb.nodesource.com/setup_15.x | bash -
RUN apt-get install nodejs -y
RUN npm install --global yarn
RUN apt-get install -y dotnet-sdk-2.1 dotnet-sdk-3.1 docker-ce
#RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add -
#RUN echo "deb https://dl.yarnpkg.com/debian/ stable main" tee /etc/apt/sources.list.d/yarn.list
#RUN apt-get update && apt-get install yarn
#RUN curl -L https://github.com/PowerShell/PowerShell/releases/download/v7.0.0/powershell-7.0.0-linux-x64.tar.gz -o /tmp/powershell.tar.gz
#RUN mkdir -p /opt/microsoft/powershell/7 && tar zxf /tmp/powershell.tar.gz -C /opt/microsoft/powershell/7 && chmod +x /opt/microsoft/powershell/7/pwsh
#RUN ln -s /opt/microsoft/powershell/7/pwsh /usr/bin/pwsh
WORKDIR /azp
COPY ./start.sh .
RUN chmod +x start.sh
CMD ["./start.sh"]start.sh (verbatim from MS)
#!/bin/bash
set -e
if [ -z "$AZP_URL" ]; then
echo 1>&2 "error: missing AZP_URL environment variable"
exit 1
fi
if [ -z "$AZP_TOKEN_FILE" ]; then
if [ -z "$AZP_TOKEN" ]; then
echo 1>&2 "error: missing AZP_TOKEN environment variable"
exit 1
fi
AZP_TOKEN_FILE=/azp/.token
echo -n $AZP_TOKEN > "$AZP_TOKEN_FILE"
fi
unset AZP_TOKEN
if [ -n "$AZP_WORK" ]; then
mkdir -p "$AZP_WORK"
fi
rm -rf /azp/agent
mkdir /azp/agent
cd /azp/agent
export AGENT_ALLOW_RUNASROOT="1"
cleanup() {
if [ -e config.sh ]; then
print_header "Cleanup. Removing Azure Pipelines agent..."
./config.sh remove --unattended \
--auth PAT \
--token $(cat "$AZP_TOKEN_FILE")
fi
}
print_header() {
lightcyan='\033[1;36m'
nocolor='\033[0m'
echo -e "${lightcyan}$1${nocolor}"
}
# Let the agent ignore the token env variables
export VSO_AGENT_IGNORE=AZP_TOKEN,AZP_TOKEN_FILE
print_header "1. Determining matching Azure Pipelines agent..."
AZP_AGENT_RESPONSE=$(curl -LsS \
-u user:$(cat "$AZP_TOKEN_FILE") \
-H 'Accept:application/json;api-version=3.0-preview' \
"$AZP_URL/_apis/distributedtask/packages/agent?platform=linux-x64")
if echo "$AZP_AGENT_RESPONSE" | jq . >/dev/null 2>&1; then
AZP_AGENTPACKAGE_URL=$(echo "$AZP_AGENT_RESPONSE" \
| jq -r '.value | map([.version.major,.version.minor,.version.patch,.downloadUrl]) | sort | .[length-1] | .[3]')
fi
if [ -z "$AZP_AGENTPACKAGE_URL" -o "$AZP_AGENTPACKAGE_URL" == "null" ]; then
echo 1>&2 "error: could not determine a matching Azure Pipelines agent - check that account '$AZP_URL' is correct and the token is valid for that account"
exit 1
fi
print_header "2. Downloading and installing Azure Pipelines agent..."
curl -LsS $AZP_AGENTPACKAGE_URL | tar -xz & wait $!
source ./env.sh
print_header "3. Configuring Azure Pipelines agent..."
./config.sh --unattended \
--agent "${AZP_AGENT_NAME:-$(hostname)}" \
--url "$AZP_URL" \
--auth PAT \
--token $(cat "$AZP_TOKEN_FILE") \
--pool "${AZP_POOL:-Default}" \
--work "${AZP_WORK:-_work}" \
--replace \
--acceptTeeEula & wait $!
print_header "4. Running Azure Pipelines agent..."
trap 'cleanup; exit 130' INT
trap 'cleanup; exit 143' TERM
# To be aware of TERM and INT signals call run.sh
# Running it with the --once flag at the end will shut down the agent after the build is executed
./run.sh & wait $!
and here is how I build and push in a kubernetes post docker world:
- task: AzureCLI@2
displayName: Azure CLI
inputs:
azureSubscription: dbazure
scriptType: pscore
workingDirectory: $(Build.SourcesDirectory)/src
scriptLocation: inlineScript
inlineScript: |
az aks get-credentials -n redacted -g redacted --admin
if ((docker buildx ls | select-string $([System.Environment]::MachineName)).length -eq 0) {
docker buildx create --name $([System.Environment]::MachineName) --driver kubernetes --driver-opt "replicas=1" --use
docker buildx inspect --bootstrap
}
$registryToken = (az acr login -n redacted --expose-token |ConvertFrom-Json).accessToken
$config = (@{auths=@{"redacted.azurecr.io"=@{username=[System.Guid]::Empty.ToString();password=$registryToken}}} | ConvertTo-Json)
$config | set-content ~/.docker/config.json
docker buildx build --push -t redacted.azurecr.io/documents-api:$(Build.BuildNumber) -f DocumentsDockerfile .Thanks I solved it. My scenario is a bit different since my agents are not K8s but you pushed me in the right direction - I had to install docker cli in the agent container.
It's funny because before the agent upgrade everything was working by simply mapping /usr/local/bin/docker into the container with
volumes:
- /usr/local/bin/docker:/usr/bin/docker
- /var/run/docker.sock:/var/run/docker.sockIf anyone faces the same problem here is the general direction:
RUN curl -fsSL https://get.docker.com -o get-docker.sh
RUN chmod +x get-docker.sh
RUN sh get-docker.shTo test if it works:
docker.compose exec dockeragent bash docker -v - if everything is OK this should return the Docker version
rreis27
commented
3 years ago Thanks I solved it. My scenario is a bit different since my agents are not K8s but you pushed me in the right direction - I had to install docker cli in the agent container.
It's funny because before the agent upgrade everything was working by simply mapping /usr/local/bin/docker into the container with
volumes: - /usr/local/bin/docker:/usr/bin/docker - /var/run/docker.sock:/var/run/docker.sockIf anyone faces the same problem here is the general direction:
- remove the docker volume mapping from compose but leave the socket bind
- install docker cli into the agent image by updating the Dockerfile, I used the official script provided on https://get.docker.com The interesting Dockerfile lines are:
RUN curl -fsSL https://get.docker.com -o get-docker.sh RUN chmod +x get-docker.sh RUN sh get-docker.shTo test if it works:
- start the agent on locally,
- connect to the container:
docker.compose exec dockeragent bash- test if docker is installed with:
docker -v- if everything is OK this should return the Docker version
I am relatively new to Azure pipelines and specifically agents, but I have my self-hosted linux agent running in a docker container. I just cannot get docker to work properly within the container. I am trying to follow how to use the socket bind, but right now I don't have anything working, docker will not start in the container as it has some issues with iptables. It feels like I'm missing something basic with socket binding, can anyone help? I gladly post my Dockerfile if that's helpful.
worldspawn
commented
3 years ago @rreis27 this expects that that docker is installed and running on the container host AND the docker cli is installed in the container. The volume bind just makes the docker cli connect to the docker daemon that is running on the host. Does that describe your scenario?
Your other option might be to use az acr build but that has no caching at all and is comparatively slow,
rreis27
commented
3 years ago Thank you @worldspawn! I found the issue as I had docker running on the container host, however it was not properly configured as I had an error in my docker systemctl file. Once I was able to resolve that my container could connect to the socket and I'm working on my first container agent build, very exciting!!!
Thank you all for the posts!
w5l
commented
1 year ago For me, the problem was caused by running a test matrix, and one of the matrix variables was called Path. I guess it conflicts with the PATH environment variable, which then breaks pretty much everything. Docker was just the first executable I called, so that's where it failed.
Bug
Type: here
Enter Task Name: Docker@2
Environment
Self hosted agent container running in Azure Kubernetes.
Agent - Private:
Issue Description
When I force a job to run on the self hosted agent (instead of azure pipelines) the docker task fails.
Task logs
Logs from the Initilise (install)