add - build script to generate improved documentation and config file for project - etcd, website

webdawg commented 1 day ago

Hello All!

I am currently implementing a new etcd stack in archlinux, and in these efforts I am using systemd config based environmental variables. In doing this the aur package in archlinux is using system.d style of configuration.

There is a /usr/lib/systemd/system/etcd.service.d/10-EnvironmentFile.conf that has a reference to the EnvironmentFile=-/etc/conf.d/etcd for the unit and a nice (but needs updated) ENV_* configuration file in the package that is /etc/conf.d/etcd

Currently I am undertaking a reformat and update of the /etc/conf.d/etcd for my work. I would like to write, eventually, a script to generate my configuration (/etc/conf.d/etcd) file based on the etcd --help output. I see there was a removal of ENV_ variables after etcd 3.4, and I can understand why. Who would want to maintain all that?

I rose this in #sig-etcd slack channel recently (https://kubernetes.slack.com/archives/C3HD8ARJ5/p1727313996375469?thread_ts=1727313558.679039&cid=C3HD8ARJ5), and I am submitting an issue here today to see if anyone thinks it may be wise to create a build script based on the output of etcd --help or similar. The build script would serve two purposes I think. The first to give folks a great .conf / ENV_ style configuration for their use, and then also generate pieces or parts of the actual https://etcd.io/docs/v3.5/op-guide/configuration/ page

How would I go about doing this? I just need the highlevel. Write script in bla (python?), submit to website repo, target X file, and Y file using these paths. Or even something like see website build script here and figure it out.

Please feel free to comment with guidance or advice.

Here is some sample work for the ENV_*/.conf file I am building right now. I am open for suggestions. For me it is a new take on the eggdrop config style work seen here: https://github.com/eggheads/eggdrop/blob/develop/eggdrop.conf

Here is a preview of what I am working on:

################################################################################
############################## Background ######################################
################################################################################

# Use this file to pull these env into a systemd service.  For instance:
# /usr/lib/systemd/system/etcd.service.d/10-EnvironmentFile.conf
#
# Contents:
# [Service]
# EnvironmentFile=-/etc/conf.d/etcd

# FYI
# Environment variables: every flag has a corresponding environment variable that has
# the same name but is prefixed withETCD_ and formatted in all caps and snake case. For
# example, --some-flag would be ETCD_SOME_FLAG.

################################################################################
############################## Member flags ####################################
################################################################################

#### --name
#
# ETCD_NAME="default"
#
# Human_readable name for this member.
#
# default: "default"
#
# This value is referenced as this node’s own entries listed in the
# --initial-cluster flag (e.g., default=http://localhost:2380). This needs to
# match the key used in the flag if using static bootstrapping. When using
# discovery, each member must have a unique name. Hostname or machine-id can
# be a good choice.

jmhbnz commented 1 day ago

Thanks for raising this @webdawg. As a maintainer of this repo in a broad sense I would be interested in some kind of automation to assist with the maintenance of these pages:

You're right - from 3.5 onwards we switched to a new layout for those pages which removed the explicit listing of the environment variables and instead just explained how the environment variables for each flag could be derived.

I think a challenge trying to implement fully automated generation of the pages will be that for some configuration flags I suspect (but haven't verified) that some configuration flags will have additional description flavour or notes added to them over and above what is currently output from etcd --help.

Perhaps a starting point for some partial automation we could begin with is a script we can run in CI that would:

compare the flags present in etcd --help for that minor version, and verify that for each flag is actually present in the website markdown file.
manipulate the file to append an environment variable value above/underneath the command line parameter for each flag.

That would address two problems we currently have where sometimes we are missing flags in the website docs which are actually present and available in the binary, and also bringing back environment variable values in a way that doesn't cause maintainer burden.

We could start with a simple script that could be run as a make recipe manually. Then if it's working nicely we could add a prow job in kubernetes/test-infra to run it as presubmit or on a schedule.

@ivanvc, @spzala what do you think about this?

ivanvc commented 1 day ago

I like the idea. 3.4's layout is more explicit/verbose, but it is also easier for end-users to reference what configuration option they are using.

We could start with a simple script that could be run as a make recipe manually. Then if it's working nicely we could add a prow job in kubernetes/test-infra to run it as presubmit or on a schedule.

Given our constraints with running automation jobs, I think the only option we have now is to run manually. We could run it along with the nightly job that updates the current version (as I would expect to run it with the released binaries).

etcd-io / website

add - build script to generate improved documentation and config file for project - etcd, website #903