search

woodpecker-ci / woodpecker

Woodpecker is a simple, yet powerful CI/CD engine with great extensibility.
https://woodpecker-ci.org
Apache License 2.0
4.07k stars 353 forks source link

Add a cookbook #316

Closed anbraten closed 8 months ago

anbraten commented 2 years ago

Add all kinds of samples with woodpecker to the documentation to give new users an idea what is possible and how common pipelines can be created:

For example:

6543 commented 2 years ago

Even better than samples are links to projects using us ... (link to conf file)

smainz commented 1 year ago

Here is one recipe I can contribute. Questions:

  1. Is there a better approach?
  2. Where would be a place to store this?
  3. Is anybody interested in a plugin which will automate the step, so you do not have to copy and paste the code in all pipelines?

Interact with git repository in your steps

Problem

Woodpecker provides a clone step for cloning the repository into the workspace but does not expose username and password for interacting with the git repository in subsequent steps. If your pipeline steps need to interact with the git repository (be it a simple git fetch, update the version of a project npm version minor or a complete gitflow release process) you have to provide a means to do so. This is how to provide all subsequent steps in the pipeline with access to the repo it was cloned from.

Recipe

  1. Create a user in the forge who has the required permissions on the repo
  2. Create a ssh key for that user and add the public key to the forge
  3. Add the private key of this user to woodpecker (either global, repo or pipeline level) using the name WOODPECKER_SSH_KEY

  4. Add this to your pipeline before the steps which require access to the git repository:

    pipeline:
  - name: Provide access to git repo
    image: woodpeckerci/plugin-git
    secrets:
      - WOODPECKER_SSH_KEY
    commands:
      # Prepare ssh key to use with git      
      - echo "$${WOODPECKER_SSH_KEY}" >  $(pwd)/.git/woodpecker.key 
      - chmod 600 $(pwd)/.git/woodpecker.key 
      # Define git user
      - git config --local user.email "woodpecker-ci@example.com"
      - git config --local user.name "Woodpecker-CI"
      - git config --local core.sshCommand 'ssh -i $(pwd)/.git/woodpecker.key -o StrictHostKeyChecking=no'     
      # Change url of origin from http to ssh
      - SSH_REPO_URL=$$(git remote get-url origin | sed -E -e "s/https?:\\/\\//git@/g" -e "s/\\//:/") 
      - git remote set-url origin $${SSH_REPO_URL}

Of course you can change the name of the secret of the ssh key, the gut user name and git user email to your taste.

Here is an example on how to interact with the repo in the next step. The only thing required is an image with git installed.

      - name: Use git credentials  
        image: woodpeckerci/plugin-git
        commands:
          - git fetch

The recipe abuses the .git repository to persist the ssh key file between different steps. The reason for this is thet .git is ignored by git and you do not get untracked file you would have otherwise to care about. Usually on would place a proper config in ~/.ssh/config, but this does not persist between steps.

anbraten commented 1 year ago

Is there a better approach?

I guess its fine. Woodpecker itself uses something similar to push the generated docs to a branch: https://github.com/woodpecker-ci/woodpecker/blob/master/.woodpecker/docs.yml#L54-L78

Where would be a place to store this?

Docusaurs (our website framework) offers to host a blog area. I think we could use it to have those guides as posts. What do you think?

Is anybody interested in a plugin which will automate the step, so you do not have to copy and paste the code in all pipelines?

A "git-setup" plugins sounds cool. 😅

gapodo commented 1 year ago

Docusaurs (our website framework) offers to host a blog area. I think we could use it to have those guides as posts. What do you think?

I'm unsure whether we should include it in the main docs, or if opening a "community" docs repo would be better.

This would split docs into "maintained by the team" and community contributions, where it's clear what gets maintained actively (matching the current state) and what just gets a "best effort" policy from the community, since maintaining a potentially huge library of templates / cookbooks can get quite taxing when changes happen (we can reference the community stuff from the main docs and also "graduate" important contributions to the main docs if we think maintaining it is worth it).

Maybe we could add something like a verified on version X table or something similar to each entry and also a last modified or similar (I know somewhat redundant with git, but going to git and checking each file history can also be quite taxing and this could be automated).

Splitting the community docs would also alleviate the pressure of X open issues and PRs on the main repo. Additionally, there would be a potential to assign additional people to the community repo if the Workload gets too high (without changing the maintainer structure of the main project).

smainz commented 1 year ago

Having a community repo would be a good starting point which would not put too much burdon on the maintainers, but it should be possible for others (not the original author) to update the docs. Whou would approve PRs and review the content?

Do you have an idea about the structure of the cookbook / recipes so I can try to put up a template structure, but be warned, I am not a good witer.

@anbraten Here is the plugin: https://github.com/smainz/woodpecker-git-setup Feedback welcome

gapodo commented 1 year ago

TBH I'm not quite sure how I would structure the cookbook either, maybe (the sub items are just there to give an idea) like this (I know that this is potentially becoming a matrix with procedures and languages potentially overlapping)?

but it should be possible for others (not the original author) to update the docs.

Absolutely, else the docs will get outdated quickly and errors (intentional or accidents) will not get fixed, a documentation like this lives of the community as a whole and not just from people initially throwing stuff in, keeping it alive and improving it will be what makes it a worthwhile reference.

Whou would approve PRs and review the content?

For the start it would likely be the maintainers, until they have decided on who they entrust with the permission to do so, potentially there may be the option to have more people on board, but also require more approvals to even out the potentially reduced trust in each of the approving parties. And potentially also add some automated testing to ensure certain conditions are met.

Since it's a "community" contribution, we need to prefix it with some warnings regarding use of unknown containers,... but a cookbook should not be able to cause too much damage as long as the users are informed (copy and paste is nice, but think about what it does before running it).

It may be meaningful to make up some conventions on how to note certain things (i.e. which urls to use for examples,...) and somewhat keep an eye on how contributions are worded (i.e. discourage comparisons where they are not needed, don't advertise other solution).

Also, we'll need to make the license agreement when contributing (CLA and DCO) to the cookbook very clear, so we don't create a hellscape of different licenses within the community docs.

Additionally, maybe a policy on how / what can be linked to and if we want to encourage links to be to a file in a specific version (permanent link) or to the project where one can see it in action.