Agent not triggered - Githubissues

cap-jmk commented 2 years ago

Component

agent, web-ui

Describe the bug

I was configuring woodpecker-ci to work with Gitea locally.

Both are deployed on two different servers.
Gitea has SSL
Woodpecker is not setup for SSL

Setup went clean.

I can connect woodpecker with `Gitea
Webhook is triggered

Yet the agent does not execute and is idle (even after the timeout interval)

The docker-compose.yml is configured according to the manual as follows:

version: '3.5'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:v0.15.3
    ports:
      - 80:80
      - 443:443
      - 9000:9000
    networks:
      - woodpecker
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
      - /var/run/docker.sock:/var/run/docker.sock
    restart: always
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_GITEA=true
      - WOODPECKER_SERVER_ADDR=:80
      - WOODPECKER_LETS_ENCRYPT=${WOODPECKER_LETS_ENCRYPT}
      - WOODPECKER_HOST=${WOODPECKER_HOST}
      - WOODPECKER_ADMIN=${WOODPECKER_ADMIN}
      - WOODPECKER_ORGS=${WOODPECKER_ORGS}
      - WOODPECKER_REPO_OWNERS=${WOODPECKER_REPO_OWNERS}
      - WOODPECKER_GITEA_URL=${WOODPECKER_GITEA_URL}
      - WOODPECKER_GITEA_CLIENT=${WOODPECKER_GITEA_CLIENT_ID}
      - WOODPECKER_GITEA_SECRET=${WOODPECKER_GITEA_CLIENT_SECRET}
      - WOODPECKER_AGENT_SECRET="${WOODPECKER_AGENT_SECRET}"

      - WOODPECKER_DATABASE_DRIVER=sqlite3
      - WOODPECKER_DATABASE_DATASOURCE=/var/lib/woodpecker/woodpecker.sqlite
      - WOODPECKER_DEBUG=true

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=woodpecker-server:9000
      - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET}

networks:
  woodpecker:
    name: woodpecker_network

volumes:
  woodpecker-server-data:

System Info

- woodpeckerci/woodpecker-server:v0.15.3
- woodpeckerci/woodpecker-agent:latest
- RaspberryOS 64bit

Validations

[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A join our Discord Chat Server or the Matrix room.

qwerty287 commented 2 years ago

Can you access the logs?

cap-jmk commented 2 years ago

Probably, but I am new to Woodpecker. How can I access the logs?

6543 commented 2 years ago

They are printed to stdout

So just docker-compose logs ...

cap-jmk commented 2 years ago

The error is somewhat logical

agent_from_doc-woodpecker-agent-1   |
 {"level":"error","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: 
Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: no such host\"","time":"2022-09-15T14:09:50Z","message":"grpc error: done(): code: 
Unavailable: rpc error: code = Unavailable desc = connection error: 
desc = \"transport: Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: no such host\""}

6543 commented 2 years ago

First i would use same tag vor server and agent, as that can cause problems otherwhise ...

And the issue you face is some sort of docker/docker-compose one. docker can not resolve the server to an ip address, you might add a hostname into docker-compose

https://stackoverflow.com/questions/29924843/how-do-i-set-hostname-in-docker-compose

cap-jmk commented 2 years ago

Good idea, yet following the suggestion yields the same error as above. Implementing the docs with localhost seems like a step forward but throws

agent_from_doc-woodpecker-agent-1   | {"level":"error","error":"rpc error: code = Unavailable
 desc = connection error: desc = \"transport: Error while dialing dial tcp 127.0.0.1:9000: connect: 
connection refused\"","time":"2022-09-17T22:09:26Z","message":"grpc error: done(): code: 
Unavailable: rpc error: code = Unavailable desc = connection error: 
desc = \"transport: Error while dialing dial tcp 127.0.0.1:9000: connect: connection refused\""}

6543 commented 2 years ago

Can you update the docker-compose at tje issue desc. to the current one

cap-jmk commented 2 years ago

It might be some socket or port issue.

version: '3.5'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:v0.15.3
    ports:
      - 80:80
      - 443:443
      - 9000:9000
    networks:
      - woodpecker
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
      - /var/run/docker.sock:/var/run/docker.sock
    hostname: woodpecker
    restart: always
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_GITEA=true
      - WOODPECKER_SERVER_ADDR=:80
      - WOODPECKER_LETS_ENCRYPT=${WOODPECKER_LETS_ENCRYPT}
      - WOODPECKER_HOST=${WOODPECKER_HOST}
      - WOODPECKER_ADMIN=${WOODPECKER_ADMIN}
      - WOODPECKER_ORGS=${WOODPECKER_ORGS}
      - WOODPECKER_REPO_OWNERS=${WOODPECKER_REPO_OWNERS}
      - WOODPECKER_GITEA_URL=${WOODPECKER_GITEA_URL}
      - WOODPECKER_GITEA_CLIENT=${WOODPECKER_GITEA_CLIENT_ID}
      - WOODPECKER_GITEA_SECRET=${WOODPECKER_GITEA_CLIENT_SECRET}
      - WOODPECKER_AGENT_SECRET="${WOODPECKER_AGENT_SECRET}"

      - WOODPECKER_DATABASE_DRIVER=sqlite3
      - WOODPECKER_DATABASE_DATASOURCE=/var/lib/woodpecker/woodpecker.sqlite
      - WOODPECKER_DEBUG=true

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=localhost:9000
      - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET}

networks:
  woodpecker:
    name: woodpecker_network

volumes:
  woodpecker-server-data:

cap-jmk commented 2 years ago

Okay, after fiddling around with the networks, I got that working up to another error message. I am not sure how to solve this yet, but I am sure I it can be fixed.

Error message

agent_from_doc-woodpecker-agent-1  

 | {"level":"error","error":"rpc error: code = Unknown desc = invalid agent 

token","time":"2022-09-19T14:56:43Z","message":"pipeline done with error"}

docker-compose.yml

version: '3.5'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:v0.15.3
    ports:
      - 80:80
      - 443:443
      - 9000:9000
    networks: 
      default:
        ipv4_address: "172.50.0.11"
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
      - /var/run/docker.sock:/var/run/docker.sock
    hostname: woodpecker
    restart: always
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_GITEA=true
      - WOODPECKER_SERVER_ADDR=:80
      - WOODPECKER_LETS_ENCRYPT=${WOODPECKER_LETS_ENCRYPT}
      - WOODPECKER_HOST=${WOODPECKER_HOST}
      - WOODPECKER_ADMIN=${WOODPECKER_ADMIN}
      - WOODPECKER_ORGS=${WOODPECKER_ORGS}
      - WOODPECKER_REPO_OWNERS=${WOODPECKER_REPO_OWNERS}
      - WOODPECKER_GITEA_URL=${WOODPECKER_GITEA_URL}
      - WOODPECKER_GITEA_CLIENT=${WOODPECKER_GITEA_CLIENT_ID}
      - WOODPECKER_GITEA_SECRET=${WOODPECKER_GITEA_CLIENT_SECRET}
      - WOODPECKER_AGENT_SECRET="${WOODPECKER_AGENT_SECRET}"

      - WOODPECKER_DATABASE_DRIVER=sqlite3
      - WOODPECKER_DATABASE_DATASOURCE=/var/lib/woodpecker/woodpecker.sqlite
      - WOODPECKER_DEBUG=true

  woodpecker-agent:
    image: woodpeckerci/woodpecker-agent:latest
    command: agent
    restart: always
    depends_on:
      - woodpecker-server
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
    environment:
      - WOODPECKER_SERVER=woodpecker:9000
      - WOODPECKER_AGENT_SECRET=${WOODPECKER_AGENT_SECRET}
    networks:
      default:
        ipv4_address: "172.50.0.12"
    extra_hosts: 
      - "woodpecker:172.50.0.11"

networks:
  default:
    ipam: 
      config:
        - subnet: "172.50.0.0/24"
          gateway: "172.50.0.1"

volumes:
  woodpecker-server-data:

wxiaoguang commented 2 years ago

I have a feeling that you forgot to put the woodpecker-server and woodpecker-agent in the same network (in your first config, you want to access host woodpecker-server from the woodpecker-agent container. However there is no network definition in your woodpecker-agent config section)

Then later, it's not correct to use docker internal IP addresses in most cases. You could mapping the ports to the host and let agent communicate with server by the host-ip:mapped-port, or use the first config to put them in the same network and use hostnames.

My docker-compose (because I deployed them in different hosts, they do not share the same docker network)

  woodpecker-agent:
    environment:
      # use the host ip and mapped port
      - WOODPECKER_SERVER=10.0.0.4:19000

  woodpecker-server:
    ports:
      - 18001:8000
      - 19000:9000

wxiaoguang commented 2 years ago

invalid agent token is related to agent-secret

bergpb commented 1 year ago

I faced a similar error here, but was related to my WOODPECKER_HOST variable, because of that the server container cannot ran as expected and shows the error in agent. After check the error message and format the host variable to the correct format, the containers started like expected.

Above the error message showed before change the variable WOODPECKER_HOST.

Attaching to woodpeckerci-woodpecker-agent-1, woodpeckerci-woodpecker-server-1 woodpeckerci-woodpecker-server-1 | {"time":"2022-10-12T19:22:39Z","message":"LogLevel = warn"} woodpeckerci-woodpecker-server-1 | {"level":"fatal","time":"2022-10-12T19:22:39Z","message":"WOODPECKER_HOST must be <scheme>://<hostname> format"} woodpeckerci-woodpecker-server-1 exited with code 1 woodpeckerci-woodpecker-agent-1 | {"level":"error","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: server misbehaving\"","time":"2022-10-12T19:22:39Z","message":"grpc error: done(): code: Unavailable: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: server misbehaving\""} woodpeckerci-woodpecker-agent-1 | {"level":"error","error":"rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: server misbehaving\"","time":"2022-10-12T19:22:40Z","message":"grpc error: done(): code: Unavailable: rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing dial tcp: lookup woodpecker-server on 127.0.0.11:53: server misbehaving\""}

qwerty287 commented 8 months ago

This is open for 1.5 years now without recent comments, so I'll close. If you still experience this please comment

woodpecker-ci / woodpecker

Agent not triggered #1187

Component

Describe the bug

System Info

Validations