Unnecessary errors in the log output of Woodpecker agent with the Docker backend using a Podman socket

IvoWingelaar commented 1 week ago

Component

agent

Describe the bug

The logs produced by the Woodpecker agent misleadingly contain unnecessary error entries when running the Docker backend with Podman.

Steps to reproduce

Install the latest Woodpecker server and agent (v2.7.3, both running in containers).
Configure the agent to use a rootless Podman socket with the Docker backend. Make sure that the minimum log level is "error" on the agent.
Run any pipeline that creates some containers with that agent. Verify that the pipeline finishes without errors in the Woodpecker server.
Verify that the environment managed by the Woodpecker agent is cleaned up.
View system log output of Woodpecker agent.

Expected behavior

I expect to see no error messages in the Woodpecker agent logs if the pipeline succeeds fully and the environment managed by the agent is cleaned up afterwards. However, during cleanup, the following output is produced:

Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0PMAND10\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:330","message":"could not remove container 'test'"}
Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0PMAND10\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:327","message":"could not kill container 'test'"}
Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0MP2RT54\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:330","message":"could not remove container 'build'"}
Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0MP2RT54\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:327","message":"could not kill container 'build'"}
Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0HXAN180\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:330","message":"could not remove container 'clone'"}
Nov 12 14:50:56 test-vm woodpecker-agent[457598]: {"level":"error","error":"Error response from daemon: no container with name or ID \"wp_01JCGDP7ZXN06SS9FE0HXAN180\" found: no such container","time":"2024-11-12T14:50:56Z","caller":"/src/pipeline/backend/docker/docker.go:327","message":"could not kill container 'clone'"}

As mentioned earlier, in my research I found the following: https://github.com/woodpecker-ci/woodpecker/issues/85#issuecomment-1836750811

I believe making the contains call in the isErrContainerNotFoundOrNotRunning function case-insensitive would fix this issue, but I do not have any Go experience to test / fix it.

System Info

Output of https://woodpecker-server/version:

{"source":"https://github.com/woodpecker-ci/woodpecker","version":"2.7.3"}

Podman version (distro package installed with # apt install):

woodpecker-agent@test-vm:~$ podman version
Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.19.8
Built:        Thu Jan  1 00:00:00 1970
OS/Arch:      linux/arm64

OS-info:

woodpecker-agent@test-vm:~$ cat /etc/os-release
PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"

Additional context

No response

Validations

[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Checked that the bug isn't fixed in the next version already [https://woodpecker-ci.org/faq#which-version-of-woodpecker-should-i-use]

zc-devs commented 1 week ago

In other words, you are requesting support for podman backend? So, duplicate? :)

IvoWingelaar commented 1 week ago

Even though I would really like a native Podman backend, I'm fine with the current state of support as mentioned in the docs: https://woodpecker-ci.org/docs/administration/backends/docker#podman-support

For a simple pipeline that doesn't do anything Podman-specific, I still expect my setup to work without producing error messages. So I do not believe this to be a duplicate issue.

zc-devs commented 1 week ago

I'm fine with the current state of support

If you was fine, there won't be the issue.

support as mentioned in the docs

"support" 🤣 It was a hack. It works until it doesn't. The same was with Forgejo/Gitea.

Will Drone plugins work with Woodpecker? How long? What is the level of support? Will Drone's runner work with Woodpecker server? How long? What is the level of support?

However, I agree with you, there is a bug in documentation.

I do not have any Go experience to test / fix it

Good time to start. Won't start, won't have.

pat-s commented 1 week ago

@zc-devs Your last comment reads quite snarky. @IvoWingelaar Only reported something he observed which doesn't seem alright, even being aware of non-native podman support. It is surely better to do so than not, and users should be encouraged to do so rather than their report being mangled into pieces (especially if it was well written in the first place).

@IvoWingelaar AFAIK nobody is actively working on extending the current podman support. So nothing will change in the near future. Contributions are always welcome, of course. As there isn't any official support for it, this is likely not a bug but more likely a FR. I'll leave it up to you to select the label.

zc-devs commented 1 week ago

nobody is actively working on extending the current podman support. So nothing will change in the near future. Contributions are always welcome, of course. As there isn't any official support for it, this is likely not a bug but more likely a FR

That is what I ~have said~ meant.

FR It is surely better to do so [make duplicates] than not, even being aware of existing ones [ checked ] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.

OK.

snarky

Maybe. It is, how it is, but true. Sadly, it often ends with that excuse. I wasn't Go dev also, nor DevOps. BTW, Agent really is a good starting point.

IvoWingelaar commented 1 week ago

I do not want to derail the topic of this issue, but I am scratching my head on this:

"support" 🤣 It was a hack. It works until it doesn't. The same was with Forgejo/Gitea.

I do not want to misinterpret this, but this is important: is Woodpecker's support for Forgejo considered a "hack"? My impression, per the docs is that the support is considered experimental ATM. Calling it experimental gives me the impression that there is at least an intention within the project towards eventual stabilization and support, calling it a hack does not give me the confidence to say the same.

I am asking because I am running Woodpecker + Forgejo, even though I am aware of Forgejo Actions. I specifically and knowingly chose Woodpecker as a CI tool over Forgejo Actions, as I'm convinced that the execution model of Actions is badly designed. I would very much prefer to be able to use Forgejo with Woodpecker in the long run.

zc-devs commented 1 week ago

is Woodpecker's support for Forgejo considered a "hack"?

No. I said it was, see explanation below.

My impression, per the docs is that the support is considered experimental ATM

Why? As I can see there was experimental warning in 2.7, but in next it is gone.

calling it a hack

There was no Forgejo, right? Obviously, no support. Then Gitea was forked softly as Forgejo. So, Woodpecker users used Forgejo as Gitea. Literally, using WOODPECKER_GITEA_* vars they pointed WP to Forgejo instead of Gitea. It worked for a while. That is what I call a hack. The same as in your Docker/Podman thing. Then Forgejo became hard fork and introduced some change, which broke Woodpecker. You can search for this bug, if curious. Woodpecker's maintainers made dicision to copy Gitea implementation, but adjusted the code accordingly using other library (dedicated to Forgejo). So, it became officially supported. Without hacks.

IvoWingelaar commented 1 week ago

Thanks for the clarification @zc-devs, I do admit to not having clicked on the next version of the website docs, but I wanted some explicit confirmation to be sure.

woodpecker-ci / woodpecker