search

woodpecker-ci / woodpecker

Woodpecker is a simple yet powerful CI/CD engine with great extensibility.
https://woodpecker-ci.org
Apache License 2.0
3.94k stars 353 forks source link

Unable to specify the backend engine using the CLI argument '--backend-engine' #3195

Open liv-io opened 6 months ago

liv-io commented 6 months ago

Component

agent

Describe the bug

When running woodpacker-agent without a configuration file but only command-line arguments, the backend engine cannot be defined. Somehow the argument --backend-engine does not work and it falls back to local.

System Info

Additional context

No response

Validations

qwerty287 commented 1 month ago

@liv-io I found a bug that could be related to this.

It's currently open as PR: https://github.com/woodpecker-ci/woodpecker/pull/3770

Is it possible for you to try this out and check if it fixes the bug? Or do you still have this issue?

liv-io commented 1 month ago

Is it possible for you to try this out and check if it fixes the bug? Or do you still have this issue?

@qwerty287 Yes, certainly. Thanks for your efforts. Please give me a few days, but I should be able to send you a feedback hopefully this week.

liv-io commented 1 month ago

@qwerty287 Your branch has already been merged into 2.6.0, right?

I quickly made the update from 2.5.0 to 2.6.0 and removed the DOCKER_HOST environment variable.

When starting woodpecker-agent it immediately exits with:

{"level":"info","time":"2024-06-15T12:09:14Z","message":"log level: info"}
{"level":"info","time":"2024-06-15T12:09:14Z","message":"no agent config found at '/etc/woodpecker/agent.conf', start with defaults"}
{"level":"error","engine":"docker","time":"2024-06-15T12:09:14Z","message":"selected backend engine is unavailable"}
{"level":"fatal","error":"selected backend engine docker is unavailable","time":"2024-06-15T12:09:14Z","message":"error running agent"}

If I add the environment variable again, the woodpecker-agent starts fine:

export DOCKER_HOST="unix://run/podman/podman.sock"
qwerty287 commented 1 month ago

Your branch has already been merged into 2.6.0, right?

Yes.

When starting woodpecker-agent it immediately exits with:

Did you start with the CLI argument?

If you don't use "standard" docker (e.g. podman) it is necessary to set an env var or the CLI argument. It only worked with DOCKER_HOST before, and now it should also work with the CLI argument and WOODPECKER_BACKEND_DOCKER_HOST.

liv-io commented 1 month ago

Did you start with the CLI argument?

Yes, I believe so. This is the command I am running.

The moment I export the environment variable:

qwerty287 commented 1 month ago

Is it possible for you to compile/run a custom binary so we can some debugging prints?

liv-io commented 1 month ago

Is it possible for you to compile/run a custom binary so we can some debugging prints?

Yes, certainly. It is a busy week for me, but I can try to build/run a custom release. Is there a branch I can try?

qwerty287 commented 1 month ago

Not yet, I'll try some things later this week but it's also busy for me ;)

qwerty287 commented 1 month ago

@liv-io Here you go: https://github.com/qwerty287/woodpecker/tree/debug-print

I only changed 3 lines.

liv-io commented 1 month ago

@qwerty287 Thanks a lot!

This is the output I get.

Let me know if I need to provide additional details.

Thanks for your help.

qwerty287 commented 1 month ago

Did you use the CLI flag in both cases? If not, can you try this?

liv-io commented 1 month ago

It seems to work without the --backend-engine docker flag.

Also when adding the environment variable (again without the --backend-engine docker flag).

qwerty287 commented 1 month ago

I mean the backend-docker-host flag.

This one is the problem. In your case, the engine is selected correctly, but it then fails because the docker backend says it's not available. It should work however.

Can you check the branch again? I added some more prints

liv-io commented 3 weeks ago

I don't quite understand. Sorry for confusing things.

Since I am not using Docker, but Podman I thought specifying the --backend-docker-host flag (or the environment variable export DOCKER_HOST="unix://run/podman/podman.sock") is a must since the path is different.

I ran a version of your latest branch/commit. These are the results:

qwerty287 commented 3 weeks ago

The problem is: The backend is correctly selected, but the docker backend says it is not available. If you set a custom host, it should actually work however… The cli parser lib does not recognise the flag somehow.

The only reason I can currently imagine is that there is some problem with the boolean flags. Can you try running the agent without all options that are not required? E.g. backend-docker-ipv6.

The only case that's interesting here is without env var, but with flag.

liv-io commented 3 weeks ago

Huh, it seems like your intuition was right. Thanks a lot for your help by the way!

I simply removed the --backend-docker-ipv6 false flag (without specifying the env var) and this is the output:

7:44AM INF log level: info
7:44AM INF no agent config found at '/etc/woodpecker/agent.conf', start with defaults
DOCKER DEBUG: docker host: 'unix://run/podman/podman.sock'
DOCKER DEBUG: cli context ok
DOCKER DEBUG: IsSet true, Str unix://run/podman/podman.sock
7:44AM ERR could not persist agent config at '/etc/woodpecker/agent.conf' error="open /etc/woodpecker/agent.conf: no such file or directory"
7:44AM INF starting Woodpecker agent with version 'dev' and backend 'docker' using platform 'linux/amd64' running up to 1 pipelines in parallel

It selects the docker backend (and the socket path is correct too).

Do you want me to test more variations of my command?

qwerty287 commented 3 weeks ago

Can you try --backend-docker-ipv6=false?

liv-io commented 3 weeks ago

Seems to work :+1: 

/usr/local/bin/woodpecker-agent --grpc-token <token> --backend-engine docker --backend-docker-ipv6=false --backend-docker-host unix://run/podman/podman.sock --backend-docker-tls-verify false --grpc-skip-insecure true --healthcheck true --hostname woodpecker-agent --log-level info --grpc-secure false
1:44PM INF log level: info
1:44PM INF no agent config found at '/etc/woodpecker/agent.conf', start with defaults
DOCKER DEBUG: docker host: 'unix://run/podman/podman.sock'
DOCKER DEBUG: cli context ok
DOCKER DEBUG: IsSet true, Str unix://run/podman/podman.sock
1:44PM ERR could not persist agent config at '/etc/woodpecker/agent.conf' error="open /etc/woodpecker/agent.conf: no such file or directory"
1:44PM INF starting Woodpecker agent with version 'dev' and backend 'docker' using platform 'linux/amd64' running up to 1 pipelines in parallel

Am I supposed to use the equal sign for all -- (double dash) options?

qwerty287 commented 3 weeks ago

You can do, that makes it more clear.

I can open an issue in the parser library to find out why this doesn't work if you think that's helpful

liv-io commented 3 weeks ago

The woodpecker-agent --help command does not mention the equal sign for any of the commands.

...
   --backend-docker-ipv6                               backend docker enable IPV6 (default: false) [$WOODPECKER_BACKEND_DOCKER_ENABLE_IPV6]
   --backend-docker-network value                      backend docker network [$WOODPECKER_BACKEND_DOCKER_NETWORK]
   --backend-docker-volumes value                      backend docker volumes (comma separated) [$WOODPECKER_BACKEND_DOCKER_VOLUMES]
...

Would you update the output or leave as is?

I can open an issue in the parser library to find out why this doesn't work if you think that's helpful

Do you think both options (with and without equal sign) should work? If yes, maybe we it is worth creating a ticket. Which parser library is being used?

qwerty287 commented 3 weeks ago

Do you think both options (with and without equal sign) should work? If yes, maybe we it is worth creating a ticket.

I need to check that too first, but I think it should work.

Which parser library is being used?

https://github.com/urfave/cli

liv-io commented 2 weeks ago

I need to check that too first, but I think it should work.

It seems adding an equal sign to all "long options" (with double dash) does not work. I can provide additional details if this is valuable.

Thanks for finding the library!