woodpecker-ci / woodpecker

Woodpecker is a simple yet powerful CI/CD engine with great extensibility.
https://woodpecker-ci.org
Apache License 2.0
3.95k stars 351 forks source link

Failed pipeline keeps being shown as running #1061

Closed JulianGro closed 6 months ago

JulianGro commented 1 year ago

Component

web-ui

Describe the bug

When a pipeline fails, it properly reports to fail to GitHub, but the Woodpecker WebUI still shows the pipeline as running.

System Info

{"source":"https://github.com/woodpecker-ci/woodpecker","version":"next-37146058"}

My agents are running the same version.

# docker-compose.yml
version: '3'

services:
  woodpecker-server:
    image: woodpeckerci/woodpecker-server:next
    ports:
      - 80:80
      - 443:443
      - 9000:9000
    volumes:
      - woodpecker-server-data:/var/lib/woodpecker/
    environment:
      - WOODPECKER_OPEN=true
      - WOODPECKER_HOST=https://ci.moto9000.moe
      - WOODPECKER_LETS_ENCRYPT=true
      - WOODPECKER_GITHUB=true
      - WOODPECKER_GITHUB_CLIENT=REDACTED
      - WOODPECKER_GITHUB_SECRET=REDACTED
      - WOODPECKER_AGENT_SECRET=REDACTED
      - WOODPECKER_ADMIN=REDACTED
      - WOODPECKER_REPO_OWNERS=REDACTED,REDACTED

volumes:
  woodpecker-server-data:

Additional context

`ci.moto9000.moe` has an AAAA record, but IPv6 is not configured on Docker.

For some reason I cannot upload anything:
https://data.moto9000.moe/m/2022-07-31_23.30.25_ci.moto9000.moe_d0c217bc1720.png

Validations

qwerty287 commented 1 year ago

Even if you reload the page?

JulianGro commented 1 year ago

Even if you reload the page?

Yes (It is actually the next day and it still shows it running)

qwerty287 commented 1 year ago

If you can access the database, can you check what's the status of the build? Column build_status in table builds

JulianGro commented 1 year ago

build_status in builds is running. The build_error column is empty on all builds as well. (Even though no build has ever succeeded on this instance.) build_status says killed on all other builds. I assume this specifies that I killed them and that the status for failing is different.

qwerty287 commented 1 year ago

Then this is a bug with the database. It wasn't updated after the build finished(though I don't know why).

JulianGro commented 1 year ago

For reference, this is using the standard settings (SQLite inside Docker)

qwerty287 commented 1 year ago

If that only occurred once, you should be able to manually change this field to success. I don't know why it didn't work. If this occurred multiple times, there's a bug in updating this column.

JulianGro commented 1 year ago

It has already occured over 4 times. For me it happens every time I run a build and it fails.

qwerty287 commented 1 year ago

That's weird... Are there any logs that might help debugging here?

JulianGro commented 1 year ago

Well I do see a lot of errors but nothing that seems related. I will turn on debug logging and trigger the issue again.

JulianGro commented 1 year ago

Here you go: https://paste.ubuntu.com/p/rrhJZVhtJH/ If you don't see anything in there, I guess I can make it fail sooner and change the loglevel to trace.

JulianGro commented 1 year ago

Hmm. I actually tried running with trace, but it doesn't find the config file in my repository anymore. Even manually specifying it doesn't work anymore. I tried going back to debug log level again but still nothing.

Any chance that the next branch got broken in the last three days?

qwerty287 commented 1 year ago

This only appears for about 3 days now? Actually there's nothing that could brake this. But yes, your docs don't contain something useful.

6543 commented 1 year ago

this is a posible glitch where if you have a running agent and just kill it, the agent can never report back that the step did fail and so it wont get updated ...

hotfix for UI -> hit cancle button, now it's cancled

long term fix is the "agent heartbeat" and similar functions ...

6543 commented 6 months ago

might be fixed at #3151

6543 commented 6 months ago

the other mentioned issue is now tracked at: #3157