Woodpecker CI does not work correctly out of the box

j4m3s-s commented 1 year ago

Describe the bug

Unstable versions of Woodpecker have been updated to 1.0+ which changes how the software works. It introduces backends (ssh, local, docker, kubernetes).

exec backend as referenced in configuration.nix doesn't exist anymore
local backend seems to need plugin-git (https://github.com/woodpecker-ci/plugin-git which can be built using buildGoPackage -- you can use gitMinimal for the clone part it's lighter than git) and apparently has issues passing the token to clone private repos (haven't investigated further)
daemon needs a configuration file (that can apparently be empty) for some things. I'm not sure if it needs to be writable.

Steps To Reproduce

Steps to reproduce the behavior:

Use services.woodpecker-.package = unstablePkgs.woodpecker-;
Errors out in the logs / the systemd service when started.

Expected behavior

I expected to use upstream package/module to work directly

Additional context

Agent and server must be of the same version otherwise it doesn't work
It makes some non-reversibles migrations to the DB so beware of your DB if you care about it ~

Notify maintainers

@ambroisie @techknowlogick @adamcstephens

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.45, NixOS, 23.05 (Stoat)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.5`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

ambroisie commented 1 year ago

Hi James!

I'd suggest you go look at my configuration to see how to configure Woodpecker, as it should be a pretty good example of setting up multiple agents with different backends.

Indeed, this should be using local instead of exec, this was a typo in the docs.
plugin-git is packaged in nixpkgs already.
I have not had any issues relating to a configuration file, I would advise you to ask this on their matrix channel to discuss it further.

The issue regarding private repositories is not due to NixOS, and should be fixed by this PR.

I'm not sure what issue you have against the current modules as they exist today, but please do provide feedback on what we could improve in our documentation.

j4m3s-s commented 1 year ago

Hey! Will take a look and get back to you shortly with the results. Thanks for answering quickly!

6543 commented 1 year ago

well private repos + local backend + window still dont work ... as netrc is not parsed on windows by default

also there was just reported regression for linux ... I'll investigate

6543 commented 1 year ago

daemon needs a configuration file

you revere to the agent ... it only has to be writable on first agent run, as it saves the ID the server assign to an agent, you can change the location with WOODPECKER_AGENT_CONFIG_FILE env var or --agent-config command argument

6543 commented 1 year ago

local backend seems to need plugin-git

yes and for nix, nix should take care of that on its own ... as the pre build binary's might not work here ?

pinpox commented 1 year ago

yes and for nix, nix should take care of that on its own ... as the pre build binary's might not work here ?

Yes, e.g. like this

Since there also is a NixOS module it might make sense to enable it there. @ambroisie you seem to be maintainer of it, would you consider it useful to add it there? There might be other parts worth porting, e.g. the bindpaths required to allow nix to run in the local backend.

Indeed, this should be using local instead of exec, this was a typo in the docs.

Should be fixed in the example of the module

ambroisie commented 1 year ago

The modules are kept lightweight and un-opinionated to avoid stepping on anybody's toes when they're trying to setup their own agents.

I don't think it would make sense to make them much more complicated, adding a services.woodpecker-agents.*.path option would be nice, but I'm not looking to make them more complicated than they currently are.

Instead, if you feel like the setup is still non-trivial, I'd recommend documenting how to setup the agents, like it has been for e.g: the GitLab runners.

j4m3s-s commented 11 months ago

And here I got some time to get back to this issue. (Can't believe it's been a month I'm sorry I didn't see the notifications :x ). I was about to write some config to add the path you just commited to the repo, 2fast4me. Thank you for all the changes!

Seems that private clone is still not working on my side (I'll work on it again). But having my own private clone works well instead!

ambroisie commented 11 months ago

I think private clone will be fixed in the next release since woodpecker-ci/woodpecker#2017 was merged into their main branch.

j4m3s-s commented 11 months ago

Once it's released I'll test and update this issue then!

NixOS / nixpkgs