Step cancelled without reason

[anbraten]

I am trying to build and publish Docker image using Dockerfile. I have a repository in Gitea. I am getting a "The step has been cancelled" error for the clone step. Please find the screenshot below.

Here is my pipeline

pipeline:
  publish:
    image: plugins/docker
    group: docker
    secrets: [ docker_username, docker_password ]
    settings:
      repo: dockerregistry/reponame
      dockerfile: Dockerfile
      tag: latest
    when:
      branch: master
      event: push

Am I missing or doing anything wrong?

Originally posted by @rajaseg in https://github.com/woodpecker-ci/woodpecker/issues/795#issuecomment-1051652122

[anbraten]

@rajaseg I have created this issue for you as it seemed to no be related to the original issue. Hope thats fine for you.

[rajasekhar-gundala]

@rajaseg I have created this issue for you as it seemed to no be related to the original issue. Hope thats fine for you.

@anbraten Thanks for creating a new issue.

[6543]

@rajaseg whitch architecture/system your agents do run under?

[rajasekhar-gundala]

@rajaseg whitch architecture/system your agents do run under?

@6543 I am running woodpecker server and Agent using docker-compose in swarm environment on Ubuntu Server 20.04 LTS

docker-compose.yml

version: "3.7"

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:latest
    volumes:
      - /localvolume:/var/lib/woodpecker/
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_HOST=https://woodpecker.example.com
      - WOODPECKER_GITEA=true
      - WOODPECKER_GITEA_CLIENT=client id
      - WOODPECKER_GITEA_SECRET=secret
      - WOODPECKER_GITEA_URL=https://gitea.example.com
      - WOODPECKER_AGENT_SECRET=agent secret
    networks:
      - caddy
    ports:
      - "8000:8000"
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET=agent secret
      - WOODPECKER_MAX_PROCS=10
    networks:
      - caddy
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
volumes:
  localvolume:
    driver: "local"
networks:
  caddy:
    external: true

Docker Version

Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:36 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:06 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Docker Compose Version

docker-compose version 1.26.0, build d4451659

[wxiaoguang]

There are different problems:

1. The canceled steps: you could look into your server/agent logs to see what error happens
2. For the docker usage, the pipeline container should have access to a docker socket, it's a complicated topic and some parts are covered by the woodpecker document.

[rajasekhar-gundala]

There are different problems:

1. The canceled steps: you could look into your server/agent logs to see what error happens
2. For the docker usage, the pipeline container should have access to a docker socket, it's a complicated topic and some parts are covered by the woodpecker document.

Please find the logs below

Woodpecker Server log

woodpecker_woodpecker-server.1.3bl2tn8kxubu@vps004    | {"time":"2022-02-26T10:39:51Z","message":"LogLevel = warn"}
woodpecker_woodpecker-server.1.3bl2tn8kxubu@vps004    | {"level":"error","error":"listen tcp: address 9000: missing port in address","time":"2022-02-26T10:39:51Z"}

Woodpecker Agent log

woodpecker_woodpecker-agent.1.ornkvohms7gy@vps004    | {"level":"error","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 10.0.1.168:9000: connect: connection refused\"","time":"2022-02-26T18:30:53Z","message":"grpc error: done(): code: Unavailable: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp 10.0.1.168:9000: connect: connection refused\""}

[wxiaoguang]

"error":"listen tcp: address 9000: missing port in address"

You server can not start correctly, the addr is incorrect. I highly suspect your config in use is not the one you posted.

woodpecker server need to listen to WOODPECKER_GRPC_ADDR (default :9000) to serve.

https://woodpecker-ci.org/docs/administration/server-config#woodpecker_grpc_addr

[rajasekhar-gundala]

"error":"listen tcp: address 9000: missing port in address"

You server can not start correctly, the addr is incorrect. I highly suspect your config in use is not the one you posted.

woodpecker server need to listen to WOODPECKER_GRPC_ADDR (default :9000) to serve.

https://woodpecker-ci.org/docs/administration/server-config#woodpecker_grpc_addr

I have added the below environment variable to docker-compose.yml

- WOODPECKER_GRPC_ADDR=9000

Also mapped below port as well

ports:
    - "9000:9000"

Please find updated docker-compose.yml below

version: "3.7"

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:latest
    volumes:
      - /mnt/woodpecker:/var/lib/woodpecker/
    environment:
      - WOODPECKER_HOST=https://woodpecker.example.com
      - WOODPECKER_GITEA=true
      - WOODPECKER_GITEA_CLIENT=client
      - WOODPECKER_GITEA_SECRET=secret
      - WOODPECKER_GITEA_URL=https://gitea.example.com
      - WOODPECKER_AGENT_SECRET=secret
      - WOODPECKER_ADMIN=admin
      - WOODPECKER_REPO_OWNERS=admin
      - WOODPECKER_GRPC_ADDR=9000 
    networks:
      - caddy
    ports:
      - "8000:8000"
      - "9000:9000"
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET=secret
      - WOODPECKER_MAX_PROCS=10
    networks:
      - caddy
    deploy:
      placement:
        constraints: [node.role == worker]
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure
volumes:
  woodpecker:
    driver: "local"
networks:
  caddy:
    external: true
    attachable: true

But I got a different error.

[wxiaoguang]

- WOODPECKER_GRPC_ADDR=9000 is wrong ......

You had better not set it. Or follow the document https://woodpecker-ci.org/docs/administration/server-config#woodpecker_grpc_addr strictly.

[rajasekhar-gundala]

- WOODPECKER_GRPC_ADDR=9000 is wrong ......

You had better not set it. Or follow the document https://woodpecker-ci.org/docs/administration/server-config#woodpecker_grpc_addr strictly.

I just removed - WOODPECKER_GRPC_ADDR=9000

Now I got the original error "This step has been canceled" and below logs for the Server and Agent.

Woodpecker Server Logs

woodpecker_woodpecker-server.1.z1mwb0917om8@vps004    | {"time":"2022-02-26T19:14:15Z","message":"LogLevel = warn"}
woodpecker_woodpecker-server.1.z1mwb0917om8@vps004    | {"level":"error","error":"stream: not found","time":"2022-02-26T19:14:16Z","message":"done: cannot close build_id 104 logger"}

Woodpecker Agent Logs

woodpecker_woodpecker-agent.1.wmj3ozqesm43@vps004    | {"level":"warn","repo":"reponame","build":"30","id":"113","error":"rpc error: code = Unknown desc = Proc finished with exitcode 1, Error response from daemon: client version 1.41 is too new. Maximum supported API version is 1.40","time":"2022-02-26T19:16:08Z","message":"cancel signal received"}

[wxiaoguang]

You can just google it ........

https://stackoverflow.com/questions/62079794/error-response-from-daemon-client-version-1-40-is-too-new-maximum-supported-ap

The problem is your Docker server is too old and woodpecker's docker client can not communicate with it correctly.

I do not know whether there is a way to force woodpecker's docker client use an old API, but you can upgrade your docker servertr

[6543]

@rajaseg ... that's #631 - if docker deamon is not compatible with the agent ... - for now it just cancle all builds - but the agent should check that on start in the first place and fail

[rajasekhar-gundala]

@rajaseg ... that's #631 - if docker deamon is not compatible with the agent ... - for now it just cancle all builds - but the agent should check that on start in the first place and fail

@6543 It would be good to support older API versions.

[6543]

@rajaseg that's mainly an issue with the upstream project we use ...

( https://github.com/moby/moby/tree/master/client ) - so it would be an issue on there side ... or we would have to write our own one completly!

[rajasekhar-gundala]

@rajaseg that's mainly an issue with the upstream project we use ...

( https://github.com/moby/moby/tree/master/client ) - so it would be an issue on there side ... or we would have to write our own one completly!

@6543 I tested with Drone, everything went well. It should work for Woodpecker as it is a clone of Drone. Correct me if I am wrong.

[6543]

@rajaseg well drone could still use an old library - is there a reason you can not update docker deamon ?

[rajasekhar-gundala]

@rajaseg well drone could still use an old library - is there a reason you can not update docker deamon ?

@6543 I just upgraded the docker daemon and everything works as expected. If someone runs important apps on Docker Swarm Cluster, it would be trouble to upgrade the docker daemon. That's why It would be good to support older api evrsions if possible.

[anbraten]

closing as the problem was detected for this case. improving the ux for other users will be done by #631