search

rancher-sandbox / rancher-desktop

Container Management and Kubernetes on the Desktop
https://rancherdesktop.io
Apache License 2.0
6.01k stars 282 forks source link

gitlab runner on wsl with rancher desktop not working #6503

Open thomas-schn opened 8 months ago

thomas-schn commented 8 months ago

Actual Behavior

We have a gitlab server for which we use local gitlab-runner.

The runner registration works as expected and the runner is shown as online.

We ran the runner on a Mac with docker-desktop and on a windows machine with wls2 and docker-desktop and in both cases the gitlab pipeline ran as expected. But when i run the pipeline on my windows with rancher-desktop, the pipeline failes with error-code 143.

My guess is that this is due to rancher desktop.

Steps to Reproduce

We start the gitlab runner with a bash-script. Here is the config for it:

#!/bin/bash

# Creates a volume to store the runner config
docker volume create local-gitlab-runner-config

# Creates a runner
docker run --rm --name local-gitlab-runner -d \
     -v local-gitlab-runner-config:/etc/gitlab-runner \
     -v /var/run/docker.sock:/var/run/docker.sock \
     gitlab/gitlab-runner:latest

# Registers the runner and stores the config in the volume
docker run --rm -v local-gitlab-runner-config:/etc/gitlab-runner gitlab/gitlab-runner register \
  --non-interactive \
  --url "<my-gitlab-server-url>" \
  --token "<my-token>" \
  --executor "docker" \
  --docker-image alpine:latest

Result

However, on my windows machine with rancher-desktop and wsl2(ubuntu2.04) the gitlab pipeline always fails with error code 143.

Running with gitlab-runner 16.9.0 (656c1943)
  on 17895e5c126d obYb94Fd9, system ID: r_ubMNFv5k2MyI
Preparing the "docker" executor 00:15
Using Docker executor with image registry.gitlab.com/gitlab-org/terraform-images/releases/1.4:v1.0.0 ...
Pulling docker image registry.gitlab.com/gitlab-org/terraform-images/releases/1.4:v1.0.0 ...
Using docker image sha256:104f99d4e97abc5ec58424692209eeb491bcbe6254668ec93793e976a333a9d3 for registry.gitlab.com/gitlab-org/terraform-images/releases/1.4:v1.0.0 with digest registry.gitlab.com/gitlab-org/terraform-images/releases/1.4@sha256:10b708737f434674e28cb1f66d997cd8cb431547a8408f347e4ca417693400df ...
Preparing environment 00:01
Getting source from Git repository 00:01
Cleaning up project directory and file based variables 00:01
ERROR: Job failed: exit code 143

Expected Behavior

After the registration was successfull, i would expect the pipeline to run succsessfull.

Additional Information

Kubernetes is disabled. WSL-Network tunnel is enabled

Rancher Desktop Version

1.11.1

Rancher Desktop K8s Version

1.27.6

Which container engine are you using?

moby (docker cli)

What operating system are you using?

Windows

Operating System / Build Version

Windows 11

What CPU architecture are you using?

x64

Linux only: what package format did you use to install Rancher Desktop?

None

Windows User Only

No response

gunamata commented 8 months ago

I could not reproduce the issue on my Windows 10 machine with Rancher Desktop 1.12.3. I have noticed you tried with 1.11.1 version. I could see the runner registered on the gitlab runner web interface too. Also, the commands worked fine when I ran outside a WSL distro in a Git bash terminal.

Can you please try with Rancher Desktop 1.12.3 which is the current latest version? If you still see issues then do you mind uploading full logs from your machine please? You can open the folder containing logs via Troubleshooting > Show logs. Please also check the box Show Logs before capturing the logs.

image

thomas-schn commented 8 months ago

Thanks for your quick reply. I updated rancher-desktop to 1.12.3 and ran into the same problem.

These are the logs from my local gitlab-runner container:

thomas@Gordon-Shumway:/mnt/c/Users/thomas/git/local-runner$ docker logs -f local-gitlab-runner
Runtime platform                                    arch=amd64 os=linux pid=7 revision=656c1943 version=16.9.0
Starting multi-runner from /etc/gitlab-runner/config.toml...  builds=0 max_builds=0
Running in system-mode.

Created missing unique system ID                    system_id=r_ZaqB8nHqUaOU
Configuration loaded                                builds=0 max_builds=1
listen_address not defined, metrics & debug endpoints disabled  builds=0 max_builds=1
[session_server].listen_address not defined, session endpoints disabled  builds=0 max_builds=1
Initializing executor providers                     builds=0 max_builds=1
There might be a problem with your config based on jsonschema annotations in common/config.go (experimental feature):
jsonschema: '/runners/0/docker/services_limit' does not validate with https://gitlab.com/gitlab-org/gitlab-runner/common/config#/$ref/properties/runners/items/$ref/properties/docker/$ref/properties/services_limit/type: expected integer, but got null

Configuration loaded                                builds=0 max_builds=1
Checking for jobs... received                       job=2096699 repo_url=<my-gitlab-url>
runner=obYb94Fd9
Added job to processing list                        builds=1 job=2096699 max_builds=1 project=304 repo_url=<my-gitlab-url>
Appending trace to coordinator...ok                 code=202 job=2096699 job-log=0-1024 job-status=running runner=obYb94Fd9 sent-log=0-1023 status=202 Accepted update-interval=3s
WARNING: Job failed: exit code 143
                 duration_s=4.077291428 job=2096699 project=304 runner=obYb94Fd9
Appending trace to coordinator...ok                 code=202 job=2096699 job-log=0-1287 job-status=running runner=obYb94Fd9 sent-log=1024-1286 status=202 Accepted update-interval=3s
Updating job...                                     bytesize=1287 checksum=crc32:45ee91eb job=2096699 runner=obYb94Fd9
Submitting job to coordinator...ok                  bytesize=1287 checksum=crc32:45ee91eb code=200 job=2096699 job-status= runner=obYb94Fd9 update-interval=0s
Removed job from processing list                    builds=0 job=2096699 max_builds=1 project=304 repo_url=<my-gitlab-url>

And these are the logs from the docker.log file in the ../rancher-desktop/logs folder


time="2024-02-21T07:43:43.605255329Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2024-02-21T07:43:43.605675638Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:43.605728050Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-02-21T07:43:43.605737413Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:43.915565033Z" level=info msg="ignoring event" container=09a5cc5eb9b9ab14b71d5af808d840ac8cf2e3d3fd0f2db45314835d64ffc1f3 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2024-02-21T07:43:43.915551954Z" level=info msg="shim disconnected" id=09a5cc5eb9b9ab14b71d5af808d840ac8cf2e3d3fd0f2db45314835d64ffc1f3 namespace=moby
time="2024-02-21T07:43:43.916193657Z" level=warning msg="cleaning up after shim disconnected" id=09a5cc5eb9b9ab14b71d5af808d840ac8cf2e3d3fd0f2db45314835d64ffc1f3 namespace=moby
time="2024-02-21T07:43:43.916353759Z" level=info msg="cleaning up dead shim" namespace=moby
time="2024-02-21T07:43:44.217445065Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2024-02-21T07:43:44.217830220Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:44.217840822Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-02-21T07:43:44.217846394Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:44.464017947Z" level=info msg="shim disconnected" id=ddbe3c64d2ef48c8f69e2a6d03d157f031823972580450fe16dac4c1cd2ec2a4 namespace=moby
time="2024-02-21T07:43:44.464042390Z" level=info msg="ignoring event" container=ddbe3c64d2ef48c8f69e2a6d03d157f031823972580450fe16dac4c1cd2ec2a4 module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2024-02-21T07:43:44.464311815Z" level=warning msg="cleaning up after shim disconnected" id=ddbe3c64d2ef48c8f69e2a6d03d157f031823972580450fe16dac4c1cd2ec2a4 namespace=moby
time="2024-02-21T07:43:44.464641817Z" level=info msg="cleaning up dead shim" namespace=moby
time="2024-02-21T07:43:46.035416281Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2024-02-21T07:43:46.035720799Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:46.035731570Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-02-21T07:43:46.035737232Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:46.256222077Z" level=info msg="shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:46.256239389Z" level=info msg="ignoring event" container=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2024-02-21T07:43:46.256534292Z" level=warning msg="cleaning up after shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:46.256887362Z" level=info msg="cleaning up dead shim" namespace=moby
time="2024-02-21T07:43:46.263926104Z" level=error msg="attach failed with error: error attaching stdout stream: write unix /mnt/wsl/rancher-desktop/run/docker.sock->@: write: broken pipe"
time="2024-02-21T07:43:46.546226117Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2024-02-21T07:43:46.546260604Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:46.546269693Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-02-21T07:43:46.546275035Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:46.741865702Z" level=info msg="shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:46.741953399Z" level=info msg="ignoring event" container=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2024-02-21T07:43:46.742217032Z" level=warning msg="cleaning up after shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:46.742644587Z" level=info msg="cleaning up dead shim" namespace=moby
time="2024-02-21T07:43:47.040440133Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
time="2024-02-21T07:43:47.040850623Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:47.040884907Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
time="2024-02-21T07:43:47.040893496Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
time="2024-02-21T07:43:47.389910094Z" level=info msg="shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:47.389971776Z" level=info msg="ignoring event" container=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a module=libcontainerd namespace=moby topic=/tasks/delete type="*events.TaskDelete"
time="2024-02-21T07:43:47.390255286Z" level=warning msg="cleaning up after shim disconnected" id=c481968a4b4d4be53d217c54d368684df9a3134b927d036b5653566841b12b6a namespace=moby
time="2024-02-21T07:43:47.390736117Z" level=info msg="cleaning up dead shim" namespace=moby

!!!! I tried the same with git-bash and suddenly the git-pipeline ran succsessfully. !!!!

gunamata commented 8 months ago

Ok, As it works in git-bash for you, it looks like the problem is within the WSL distro. @Nino-K , Do you have any insights to share based on the log messages shared above?