search

rsyslog / rsyslog-doc

documentation for the rsyslog project
Other
99 stars 250 forks source link

v8-stable still uses temporary version number #535

Open rgerhards opened 6 years ago

rgerhards commented 6 years ago

@deoren I experimentally merged master into a copy of v8-stable and rebuild it. However, the version number was still the temporary one. I assumed that the stable build read the version number from a file. I just had a quick glimpse, but that was not the case.

If I am right, I suggest we

That we we need to update these files only on release and can be sure that whenever someone builds from v8-stable the version numbers are pinned.

As a tangent, I would like to shorten the temporary version number used at the rsyslog.com site, it requires a lot of space. Do you have any concerns in this regard? If so, let's open a new tracker for the discussion.

deoren commented 6 years ago

@rgerhards The current configuration is using a similar setup to what you've described here. I know you've got a lot going on, but please carve out some time to read over the first part of this doc:

https://github.com/rsyslog/rsyslog-doc/blob/master/BUILDS_README.md

and my summary of the changes here:

https://github.com/rsyslog/rsyslog-doc/pull/493#issuecomment-359163281

Because you showed a preference for using the release_build.sh script to generate the docs and release tarball at release time, that script was updated to provide the required results.

@rgerhards: As a tangent, I would like to shorten the temporary version number used at the rsyslog.com site, it requires a lot of space. Do you have any concerns in this regard? If so, let's open a new tracker for the discussion.

As you noted, let's discuss on a separate issue. I'll create one so that we can cover that topic there.

rgerhards commented 6 years ago

Ok I need to re-read it. That means I need to change the doc update process so that I now find the latest tarball (I think). Right now I just do a gut checkout and be done.

rgerhards commented 6 years ago

I posted the current update script in the other tracker.

deoren commented 6 years ago

@rgerhards: That means I need to change the doc update process so that I now find the latest tarball (I think). Right now I just do a gut checkout and be done.

I think it would help to illustrate.

Starting at line 66 and on through line 70 is where the release build number logic is used to pull the version and release string details that are used later in the substitution process:

https://github.com/rsyslog/rsyslog-doc/blob/a66e62366d213d1cde4f647ff087463fe8730421/release_build.sh#L66

Starting here and continuing until line 118 is where the placeholder values are substituted with the latest stable tag values:

https://github.com/rsyslog/rsyslog-doc/blob/a66e62366d213d1cde4f647ff087463fe8730421/release_build.sh#L100

The result is a tarball that is "release build ready" for uploading to the website for others to download and for downstream package maintainers to build for inclusion in their own repos.

The build directory is then ready for you to sync to the website and contains stable release numbers in the doc version/release string. For example, rsyslog 8.33.0 documentation vs the long dev build string that our dev builds currently have.

All of that said, we can apply this same logic to the script you're using now if you wish to have a script for on-demand builds and another for release builds. If you can carve out the time, give the release_build.sh script a try with a test clone of the repo to see what I mean.

We can also extend the release_build.sh script to take command-line options to control its behavior. Perhaps if it's being driven by another script it could omit the tarball generation step and just generate the bare docs with the stable release string (e.g., rsyslog 8.33.0 documentation).

rgerhards commented 6 years ago

Pls have a look at the script *I" posted. I am not talking about the tarball generator but the web update script. It's in the other tracker and painful to copy it over with my current device. If in doubt, wait till tomorrow and I then post a copy.

deoren commented 6 years ago

@rgerhards: Pls have a look at the script *I" posted. I am not talking about the tarball generator but the web update script

@rgerhards I saw it. Sorry if I gave you the impression that I hadn't.

Let's ignore the scripts for a moment and talk about the goals. I'll post what I think you want, you can post (at your convenience) and correct what I have wrong.

The first goal I think is met through the release_build.sh script. The second and third goals it sounds like you're solving bay way of the script that you posted to https://github.com/rsyslog/rsyslog-doc/pull/533#issuecomment-362796710 and previously elsewhere.

for branch in v5-stable v7-stable v8-stable master;

It appears that your goal is straightforward: build all branches and have the release string reflect the stable version for that branch, with the exception of master. For that branch, the logic you mentioned would apply.

The current state of the docs assumes that release builds, both the tarball uploaded to the website and the v8-stable branch were generated using a source/conf.py file modified from the release_build.sh script. The current doc state also puts the focus exclusively on the v8-stable and master branches and makes no effort to modify the release string for the v5-stable or v7-stable branches.

Regarding the release string for v8-stable:

https://github.com/rsyslog/rsyslog-doc/blob/a66e62366d213d1cde4f647ff087463fe8730421/tools/buildenv/tools/build-doc#L17

From what I can tell, you've moved to using a Docker image for doc builds. The script you're calling from within the Docker image works directly from the branch without making any modifications. It also uses the Git sources for the work, not the tarball. From what I can tell this is by design and so modifying that setup to use a tarball instead could work, but it won't help you in the long run unless you're just doing a CI check to make sure that the release tarball doesn't regress.

For regular branch builds, support is needed to use stable release numbers for the v8-stable branch. This means that we auto-calculate that value at build time (my personal preference) or a flat-file is used as you suggested. I recommend that we do not use flat-files for builds from the Git repo since it is something else we have to remember to maintain and could fall out of date. Instead, let's see how we can automate the release string calculations.

The Docker image uses ash in place of bash, so that requirement will have to be met.

Let me take a stab at this and I'll share my work. If it doesn't look feasible, then going with flat-files can work. If that is the direction we go, the current source/conf.py and source/conf_helpers.py files will need to be updated to reflect the move back to flat-files.

Lastly, let's consider adding your website update script to the repo in abstract form. Where you run rsync to copy the content over you could use dummy values in the script with an include reference to overwrite them with real values for your infrastructure. This will help make it clearer to anyone that works with the build related scripts in the future what requirements are in place.

deoren commented 6 years ago

Addendum to last comment:

Sphinx supports specifying the version and release variables via command-line, so if the script can generate the desired values before running the Docker image (set via environment variables), then the build command in the image can take advantage of that.

Example:

-sphinx-build $STRICT  -b $FORMAT $SPHINX_EXTRA_OPTS source $OUTPUT
+sphinx-build $STRICT -D release="$DYNAMIC_RELEASE" -D version="$DYNAMIC_VERSION" -b $FORMAT $SPHINX_EXTRA_OPTS source $OUTPUT
deoren commented 6 years ago

Proposed script

@rgerhards I took a stab at modifying the script you posted and the result is 9ecfa593ab8836ef6846ba034d8e45613fa2df0a.

Link: https://github.com/deoren/rsyslog-doc/blob/i535-docker-stable-build-autogen/tools/official_docs_cron_script.sh

Proposed next step

It works as-is, but will need some work to include details for your prod environment. The approach I'd recommend is to have this script check for an include file in a specific location. If found, it is imported and would override the stock placeholder values with real values for your build box.

What I did

I skipped going with an approach that would have involved modifying the Dockerfile (and resulting Docker image), and instead took advantage of the $SPHINX_EXTRA_OPTS environment variable that you already had in place. This version of the script appends to that environment variable what is needed to override the default behavior in the current v8-stable branch.

I abstracted some previously hard-coded values to aid in testing, but did not seek to change too much more than that in order to keep the feel of the original script. The name of the cron script that I added is also overly long, but mainly for illustration.

Important bits

Some points for the script that I'd like to emphasize:

Other thoughts

Even once PR #536 goes through, the sed calls this script makes may still be needed for the older branches unless you're up for back-porting the absolute minimum changes needed to the source/conf.py file for those branches.

For now, (at least with this script) I think you're probably still better off with the sed calls. I'll keep this in the back of my mind and will chime in with a better solution if/when I have it.

For now, I still believe that PR #536 has value and will allow others to build the docs in a format equivalent (other than banner ad) to the official site.

rgerhards commented 6 years ago

I think we have at least two, maybe three root issues behind this conversation. They are probably the root of a couple of misunderstandings. Just let me try to describe it very briefly -- I'll provide more in-depth info a bit later (need to think about the best transport vehicle, as I would probably need hours to write this all up).

In essence, I see three very different projects/task/admin domains:

To me, these are three totally different things, each of which needs to be targeted separately. I have the impression that for you this seems to be a single thing that we can and should target in an integrated way.

I can't see which approach is better, but my software engineering education and experience tells me they should be seperated, but be able to play together modularly. Maybe we need broader feedback.

Again, I'll post a more in-depth description later.

rgerhards commented 6 years ago

OK, l've gone hardcore on you ;-) For the first time in 5 years (I noticed) I have fired up Hangouts on Air and recorded the first part of what I have to say. It's directly related to the previous message:

https://youtu.be/t40y4nfGPGE

Hangouts has cut off the first 2 seconds or so ... such is life. If you have a too-hard time with my accent, or do not find it useful in other ways, please let me know. I will not be offended. I am trying to find a better way to provide a complex thing without writing a day or so. If this try does not work out ... then we have at least tried it. Let's drop it then and resort back to writing on the tracker or something else.

rgerhards commented 6 years ago

I forgot to apologize for pressing you to viewing a video ;-) I myself hate it to watch videos for technical things, but I think in this case there really is a good reason to do so. It's a complicated best that we did not yet manage to find common ground, and the ability to explain with voice and drawings and pointing IMHO helps. Just wanted to assure you that this will definitely not become my goal for regular conversations.

deoren commented 6 years ago

@rgerhards Not a problem. My Sunday mornings (it's 12 pm for me right now) are usually dedicated to sysadmin work (server patching at the moment related to Spectre / Meltdown), so I'm busy with that. I'll definitely take a look at this and other items we've been corresponding on, I just am tied up at the moment.

In general though I'm fine with video, audio, email, or whatever other medium we need to use. As I see it you're just being flexible and are trying to make sure we get on the same page. It is appreciated, even if I don't always remember to say so.

deoren commented 6 years ago

@rgerhards Watching the video now. I find it helpful, as I do the previous comment where you lay out the perceptions on both of our parts.

You talk about interfaces, particularly an interface between the doc project and the doc project users. In general, understand what you're getting at.

You mention that users should not need Git on their systems to build the docs. I agree. In the not so distant past I did tie the build process to Git, which in hindsight is an obvious mistake. I (again) had the mistaken idea that anyone building the docs would most likely do so only as a contributor and would otherwise use the website to view current docs. This was of course a faulty view.

The current state of the docs does not require Git to build and hasn't since #493 was merged (less than a month, hence the not so distant past reference).

@rgerhards: In essence, I see three very different projects/task/admin domains:

doc project itself (w/o rendering other than w/ default theme) rendering of doc project by "users" (ReadTheDocs, Distros, rsyslog.com(!)) providing a consistent build environment (Docker)

I can see that, though the first two are very closely tied together in my view. I have a hard time describing it, but I see the doc project as a common base with many consumers. There is the end user, which can build the docs if they wish, though for casual users (of which I was for about the first 8 months of last year) they will just use the website and the mailing list (or I did anyway) for help.

For anyone that wishes to contribute, that is where the user will begin to build the docs. This is where they'll arrive at the project repo, see the README displayed as the "index" or entry-point and begin exploring. I see this as the "wrapper" (think of the quick How to Cook directions on something you're considering at the market) and requires at least some basic coverage sufficient to get someone started. IMO, it also needs a faster turnaround at times for changes than a standard development cycle, so it's useful to have some beginner Getting Started material there in Markdown files.

Once they've read over the "wrapping", they'll dig in and fetch the sources. As you (and others) noted, there is no definitive approach for what they'll use. They theoretically could grab the sources from the main site at this point, but it's just as likely that they'll download a zip or tar.gz file from the GitHub releases page (or even the master branch listing). This (for consistency I'll call all of them tarballs) file should allow users to build without Git present. I'm 100% on board with this concept and will not make the same mistake again.

When they start the build process, it should just work, provided that they have the basic dependencies for doing so in place. This is where the README comes in. The current version directs them through installing them through the requirements.txt file, which in my view is a nice handy "button" to pull in anything needed to produce the desired result. Ideally it would be a small list (and it currently is), so we shouldn't expand the file without cause. For example, if we're testing new themes and wish to look at a half dozen of them, we should not use the requirements.txt file for our benefit. I get that. I think we should however endeavor to give them the same experience that they're getting from the website version. If you believe that the new theme offers a much improved mobile experience (and I happen to agree), then the docs that the end user generates should match. In this case, if the user attempts to build the docs and they've not followed the directions and ran pip install -r requirements.txt, the build will likely still work with PR #536 in place. This way they'll end up with the classic theme applied.

With the theme applied, they get the benefits of reading the docs locally (or on their own site if they wish) with a theme you feel provides a better mobile experience. I see no downside here because of the graceful fallback. If instead the process were fragile, I wouldn't want to jeopardize the build process by mixing concerns. In this case I see it as a win for both: rsyslog.com website, end users. If this were ONLY to benefit the website, then I'd say mixing the two would go against the "separate domains" concept you describe.

@rgerhards: To me, these are three totally different things, each of which needs to be targeted separately. I have the impression that for you this seems to be a single thing that we can and should target in an integrated way.

I mentally see the docs as a base, though what picture I would use to describe it I'm not sure. I see the Docker image as an optional tool because no matter how useful it is, I can easily install Python on Windows (or built-in on other OSes), run pip install -r requirements.txt --user and I'm ready to run Sphinx to build the docs. With Docker I have to jump through some more hoops and for Windows I had a very poor experience trying to get Docker for Windows to use an existing Hyper-V setup that I had in place. I just gave up and installed Ubuntu/Docker in a Hyper-V VM. That's fine because I'm willing to do that to learn Docker, but for some that is mixing two entirely different goals. To clarify, I see the Docker image as a useful tool, but not an essential one. It's the volume control for the TV, it's not the volume switch or even the TV itself. It's handy, but again, not essential.

The doc project of course is the core piece of the topic here and the work done there and the goals for future coverage and expansion needs to be treated with respect and with a caretakers eye. Mixing in any dependencies that are not needed is harmful to the process. While I meant well with the Git requirement, I failed to take in the larger scope of the project and at the time was only thinking of the core contributors who would be maintaining it, not downstream maintainers who would be building the docs outside of the development environment that the doc team would be using. I foolishly thought of the release tarball as the source for downstream packaging efforts and that everyone else would be using the live website or the official docs release tarball (containing pre-built files). As I mentioned that assumption has been well corrected by this point.

The rsyslog.com website I see as a service for users. It's the compiled version of the docs that they can browse and use for configuring and working with the rsyslog software itself. I see this as the key vehicle for delivering the docs to the users. If it's possible to serve the needs of the site without affecting the quality of the docs for users who wish to consume the material directly, I think we should provide an interface for doing so.

Some (unorganized) things that relate to that idea:

deoren commented 6 years ago

You mention that we shouldn't force a theme on a user. I see that. PR #536 attempts to use the better theme and then falls back to the default theme if not told otherwise. The original source/conf.py file explicitly sets a html_theme option, so anyone wishing to use a different theme has to override it.

I double-checked myself to make sure I was recalling correctly and confirmed that using the -D option to override a setting in source/conf.py does indeed override the value. From the sphinx-build docs doc:

-D setting=value

Override a configuration value set in the conf.py file. The value must be a number, string, list or dictionary value.

For lists, you can separate elements with a comma like this: -D html_theme_path=path1,path2. For dictionary values, supply the setting name and key like this: -D latex_elements.docclass=scrartcl. For boolean values, use 0 or 1 as the value.

I suspect that is exactly what Read the Docs is doing for their builds. I'll try to confirm that for sure as well as double-check again the results from the changes on PR #536.

deoren commented 6 years ago

@rgerhards On my last remark I mention the use of the -D command-line option to sphinx-build. I suppose if you wanted to keep the site theme entirely separate from the default theme (even with the benefits I think having them the same would bring), then you could do as you're doing now with one exception: instead of sed calls, replace them with -D sphinx-build options to specify the values on the command-line. That should be MUCH more reliable then maintaining search/replace patterns on a moving target (i.e., the source/conf.py file).

If you do wish to have the same theme in both places (but with a graceful fallback as I've mentioned), then going with PR #536 is the way to do it (combined with updating your build process to use the newly modified requirements.txt file). As I mentioned I think there are positive reasons for using the better theme as the new rsyslog-doc project theme. Even if you ignore the use of that particular theme, I think it's important to have the default match what is used on the site, if for nothing else the reason of consistency. I stress the word default however as I also feel it's not a good idea to force the choice as it interferes with the user (directly) and downstream (maintainers and eventually the users themselves).

On the note of maintainers, it's very likely that they'll use a "clean room" build process that would not have all modules available. If they build without referencing the requirements.txt file, then they will still get good results using the changes as per #536.

deoren commented 6 years ago

@rgerhards I closed PR #536 with some notes that indirectly refer back to this ticket.

deoren commented 6 years ago

Just finished the video. How ironic that something you intended to save time should devolve into me writing pages of text as a reply. ;)

Assuming that I didn't completely miss the point, I've closed PR #536 based on the potential for causing problems with user-specified themes. My initial testing did not uncover any issues, but when you mix in overrides you have to also override the html_theme_options setting as well, which may not be something that downstream would initially know to do.

Circling back to the original topic, I still like the idea of having automated generation of release strings for dev builds and for release builds.

Here is a modified version of the stub script you posted before:

https://github.com/deoren/rsyslog-doc/blob/i535-docker-stable-build-autogen/tools/official_docs_cron_script.sh

Once you swap out the sed calls for sphinx-build -D options to override default options (in other words, html_theme and html_theme_options), install the theme via pip (so it's known to Sphinx) and update the paths in the script, I think you can build using that script. Anyone building the docs directly via sphinx-build -b html source build will get solid results, with or without Git present.

I'm going to hit pause here as I think I may be circling myself with some of my comments. Feel free to take a break before circling back to this ticket, I won't be offended.

rgerhards commented 6 years ago

If this were ONLY to benefit the website, then I'd say mixing the two would go against the "separate domains" concept you describe.

OK, I think we reach common ground here. If you really think there is value in using the "better" theme for everyone, it makes sense to provide that as a base. Quite honestly, for me it started out as, and still feels like, a compromise to get the doc integrated in a somewhat decent way with the rsyslog site. IMHO I would not use that theme if I either would not have that integration need or had more time to do it really well. So it's not something I am fond of and it's also not something I would really think is worth recommending as a great solution. IMHO it works sufficiently well for what we need and with what time we can invest, but it's far from "best practice". Probably I always falsely assumed that was obvious (sooo many chances for misunderstandings).

More later, still reading (and not sure I can finish all today - so pls be patient).

rgerhards commented 6 years ago

I see the Docker image as an optional too

Me, too. There are situations where it makes sense, and some where not. If you are python-literate, it does not make much sense for an end user. If not, the docker container is a handy way to get everything going without need to fiddle for anything. And I admit I do not care for Windows at all ;-)

The container comes in as an important tool as far as CI is concerned. For doc contributors probably not that big an advantage, but on the main project is it immensely helpful to be able to reproduce the exact same failure in your environment you have in CI.

The doc project of course is the core piece of the topic here and the work done there and the goals for future coverage and expansion needs to be treated with respect and with a caretakers eye. Mixing in any dependencies that are not needed is harmful to the process. While I meant well with the Git requirement, I failed to take in the larger scope of the project and at the time was only thinking of the core contributors who would be maintaining it, not downstream maintainers who would be building the docs outside of the development environment that the doc team would be using. I foolishly thought of the release tarball as the source for downstream packaging efforts and that everyone else would be using the live website or the official docs release tarball (containing pre-built files). As I mentioned that assumption has been well corrected by this point.

That's something you need to get used too. After all the years, I am sometimes still surprised where users get their sources from ;-) While I can share that experience, there is nothing like experiencing it oneself -- often it's too hard to believe otherwise.

While the official website build script or even the release_build.sh script may not fit the core purpose of the doc project (hold the sources + provide minimum requirements to build those sources), I believe it will be useful to contributors (part of our user base) to have them available. Perhaps we move them aside into the contrib and tools directories, but having them in the repo alongside the sources may help contributors integrate future changes to the handling of the docs re the build process.

There are some details in the scripts I do not want to expose without extremely good reason because they involve secrets. Not talking "security by obscurity" here, but not making it easier than necessary for attackers.

The docs portion of the website is the primary contact point with the docs. If there is something that can make displaying the content there more reliable or accessible to the user base (I keep wanting to say "our" users, but that may be a little presumptuous), it's probably worth at least a hard look at applying directly to the core doc project itself. The better theme is an example. If you made the decision that you want the website to have a distinct look, that changes things.

Actually IMHO it really is only about "wanting the website to have a distinct look". The now default theme - alabaster - is a way better choice, but we do not have the resources to integrate this with the web site. So we made a compromise that makes the site render OK, but this is definitely not what I would recommend to someone as a great theme (for using the doc by himself).

I think it's important to have the default match what is used on the site, if for nothing else the reason of consistency. I stress the word default however as I also feel it's not a good idea to force the choice as it interferes with the user (directly) and downstream (maintainers and eventually the users themselves).

Note sure if this is really vital. If so, I agree the "better" theme needs to go in. @davidelang do you have an opinion on this?

On the note of maintainers, it's very likely that they'll use a "clean room" build process that would not have all modules available. If they build without referencing the requirements.txt file, then they will still get good results using the changes as per #536.

I would strongly recommend that maintainer should use the docker container, as this is what CI uses and so is the official environment in which the doc must build. That's also a reason the container def is not in a separate project. aka "If it doesn't build in the container, it is broken" (actually, this is what CI does!).

Circling back to the original topic, I still like the idea of having automated generation of release strings for dev builds and for release builds.

That's fine, but the scripts for release builds need to leave all necessary artifacts so that a casual sphinx-build by the (reference!) container provides the desired output. I would still suggest to store the artifacts inside files. It's dumb simple, but that's exactly why it works well.

As a side-note, we may want to consider having a Hangout if we can't find common ground rather quickly now. It's better to chat at inconvenient time for half an hour to an hour than to keep on writing (took me roughly an hour just this morning, and I guess you at least another one or two just yesterday ;)). I admit I run out of time. Already need to shuffle most of the 8.33 main project release goals to 8.34. Not just because of this discussion, but the whole site integration theme has taken 3+ days now. The containerization work took quite some work and is also unfinished. We made some progress on the packages, but there are still miles to go. The doc is definitely in a better shape, but if there is one thing in doc, I would really like to fix all those nasty things I have seen so prominently the past two weeks. Lot's of things compete for time. Note that I do not want to complain. It's a useful discussion, and we need it. But each time I work on the site, or on the doc, or on CI, I can not work on developing the main project, or fixing bugs in libraries, or... It's simply so that any of these activities is useful, and should be done, but doing one excludes progress on all others (just for the moment, of course, I am using a round-robin scheduler). Again, I am happy to have the discussions, but I would like to make clear the restrictions we have - unfortunately :-(

rgerhards commented 6 years ago

@deoren I needed to edit my last posting for clarity. Do not rely on the mail notification. Sry for that.

deoren commented 6 years ago

Here is where I think we are:

I think we're close to wrapping this up. No rush on reading this, just whenever you need something to fall asleep trying to read. ;)

I'd like to cherry pick from recent conversation and make sure I respond to a few points, to clarify if nothing else.

Website build script

@rgerhards: There are some details in the scripts I do not want to expose without extremely good reason because they involve secrets. Not talking "security by obscurity" here, but not making it easier than necessary for attackers.

Understood. For the specifics, I was thinking that those could be pulled in from a separate include file to override some defaults that allow the script to be tested/maintained over time. I wasn't suggesting that inner workings be exposed in a public repo. I understand however if you don't have an interest in this. If we can get a close enough approximation of the script in the repo as a reference however (or even another repo) I think it would prove useful as an example of what requirements any future automation must support (aside from the understanding that things "just work" for users building the docs).

Maintainers building the docs

@deoren: On the note of maintainers, it's very likely that they'll use a "clean room" build process that would not have all modules available. If they build without referencing the requirements.txt file, then they will still get good results using the changes as per #536.

@rgerhards: I would strongly recommend that maintainer should use the docker container, as this is what CI uses and so is the official environment in which the doc must build. That's also a reason the container def is not in a separate project. aka "If it doesn't build in the container, it is broken" (actually, this is what CI does!).

Sorry, I wasn't specific enough. I meant maintainer as in "downstream" maintainer for distros that consume the tarball for doc builds. They may have their own infrastructure requirements and be unwilling to modify them to build the docs. For my part, I hear you loud and clear. I will try to work in using the Docker image as part of my testing process to help catch any potential breakage.

Doc theme

Regarding the doc theme, I think we're on the same page now. The main website is just another consumer (aka, "user") of the docs and will apply a separate theme (or at least reserve the option to apply a separate theme) just like any other group. This will likely be done via overriding the Sphinx build configuration file (source/conf.py) options using the -D command-line option.

While I think it is useful to have the theme match in both places for a consistent look and feel, this is realistically a very minor point and not something that has to be resolved in tandem with other concerns (note to self: this is why technicians and programmers prefer separate tickets with a very specific focus on each). I say let's leave the better theme in place for the main site with the understanding that if we all had our way, we'd have a clean theme to apply for both the site and the off-line docs so that the "feel" of the docs was consistent across the board.

Automated builds using stable and dev build strings

Recent remarks

@deoren: Circling back to the original topic, I still like the idea of having automated generation of release strings for dev builds and for release builds.

@rgerhards: That's fine, but the scripts for release builds need to leave all necessary artifacts so that a casual sphinx-build by the (reference!) container provides the desired output.

Situation right now

Requirements

@rgerhards: The guideline here is that running the reference container should produce the desired result.

The logic surrounding stable release builds was based around you using the release_build.sh script. I got this impression from discussion on #468 and tangent tickets. Since the guideline has shifted to using the reference container image, one approach would be to update the release_build.sh script to "wrap" the call to the docker command-line tool in order to pass n the needed options to the reference container. This could work. This of course assumes that you have plans to keep the release_build.sh script.

What constitutes an "official" stable release?

From your original post:

@rgerhards: That we we need to update these files only on release and can be sure that whenever someone builds from v8-stable the version numbers are pinned.

That is the part that I think I had the wrong impression. I had it in my head that the release tarball available from the rsyslog.com website represents the "official" stable release sources (including compiled content as a courtesy for users).

Downstream would build from those sources for inclusion in their package repos.

The tarball would be generated from you running the release_build.sh script along with the generation of an udated copy of the v8-stable branch that would be similarly tagged with a current stable release string (e.g., rsyslog 8.33.0 documentation).

Aside from the v5-stable and v7-stable branches (which I thought were both no longer maintained no longer actively built) which have hard-coded rsyslog vX-stable documentation release strings, I considered that anything else would by default be a "dev" build, representing pre-release and potentially unofficial content.

If you instead consider the v8-stable branch itself to be official and wish to have any builds generated from that branch "official" and with a release build string already in place then that definitely changes things. If that's the case, I'll need to mentally chew on the problem. If I can't resolve it within a short amount of time I will do exactly as you suggest and revert to hand-maintaining the version info within the source/conf.py file itself. The support for trigger dev build release strings will then have to be used on an opt-in basis via command-line overrides from a wrapper build script (container or not).

This really hinges on what you consider "official" and in need of a stable release string.

For my part, I only consider the website and anything built directly from the official stable release tarballs (hosted on rsyslog.com, unmodified builds provided by distro package maintainers, etc) as truly official. In my mind I believe everything else should be tagged with a "dev" identifier of some sort to help emphasize that point.

That's just me though, so I understand if you'd like to go in another direction and will update the build process accordingly to match what you outline.

rgerhards commented 6 years ago

Website build script @rgerhards: There are some details in the scripts I do not want to expose without extremely good reason because they involve secrets. Not talking "security by obscurity" here, but not making it easier than necessary for attackers.

Understood. For the specifics, I was thinking that those could be pulled in from a separate include file to override some defaults that allow the script to be tested/maintained over time. I wasn't suggesting that inner workings be exposed in a public repo. I understand however if you don't have an interest in this. If we can get a close enough approximation of the script in the repo as a reference however (or even another repo) I think it would prove useful as an example of what requirements any future automation must support (aside from the understanding that things "just work" for users building the docs).

Our thoughts crossed. See the work on https://github.com/rsyslog/rsyslog-website

Maintainers building the docs @deoren: On the note of maintainers, it's very likely that they'll use a "clean room" build process that would not have all modules available. If they build without referencing the requirements.txt file, then they will still get good results using the changes as per #536.

@rgerhards: I would strongly recommend that maintainer should use the docker container, as this is what CI uses and so is the official environment in which the doc must build. That's also a reason the container def is not in a separate project. aka "If it doesn't build in the container, it is broken" (actually, this is what CI does!).

Sorry, I wasn't specific enough. I meant maintainer as in "downstream" maintainer for distros that consume the tarball for doc builds. They may have their own infrastructure requirements and be unwilling to modify them to build the docs. For my part, I hear you loud and clear. I will try to work in using the Docker image as part of my testing process to help catch any potential breakage.

I am not really qualified for downstream maintainers, but I guess they do not want to auto-install anything but closely control what they have. I might be a good idea to throw a focussed question at user mbiebl and whissi (intentionally not mentioning them here - they for sure will not like to dive into this discussion at this point ;-)).

Rest follows, maybe not today.

rgerhards commented 6 years ago

Requirements @rgerhards: The guideline here is that running the reference container should produce the desired result.

The logic surrounding stable release builds was based around you using the release_build.sh script. I got this impression from discussion on #468 and tangent tickets. Since the guideline has shifted to using the reference container image, one approach would be to update the release_build.sh script to "wrap" the call to the docker command-line tool in order to pass n the needed options to the reference container. This could work. This of course assumes that you have plans to keep the release_build.sh script.

What constitutes an "official" stable release? From your original post:

@rgerhards: That we we need to update these files only on release and can be sure that whenever someone builds from v8-stable the version numbers are pinned.

That is the part that I think I had the wrong impression. I had it in my head that the release tarball available from the rsyslog.com website represents the "official" stable release sources (including compiled content as a courtesy for users).

Downstream would build from those sources for inclusion in their package repos.

The tarball would be generated from you running the release_build.sh script along with the generation of an udated copy of the v8-stable branch that would be similarly tagged with a current stable release string (e.g., rsyslog 8.33.0 documentation).

Aside from the v5-stable and v7-stable branches (which I thought were both no longer maintained no longer actively built) which have hard-coded rsyslog vX-stable documentation release strings, I considered that anything else would by default be a "dev" build, representing pre-release and potentially unofficial content.

That's the original intent, and how it works in an ideal world. Practical experience shows it doesn't always happen like this in the real world. I learned this on the main project. And, if so, we can benefit from it ourselfs. Building from git is far more convenient for the web site than finding out the right tarball to build.

If you instead consider the v8-stable branch itself to be official and wish to have any builds generated from that branch "official" and with a release build string already in place then that definitely changes things. If that's the case, I'll need to mentally chew on the problem. If I can't resolve it within a short amount of time I will do exactly as you suggest and revert to hand-maintaining the version info within the source/conf.py file itself. The support for trigger dev build release strings will then have to be used on an opt-in basis via command-line overrides from a wrapper build script (container or not).

can't we "just" keep the version strings inside a file, then we can check in the conf.py log if that file exists and, if so, use the version from there?

deoren commented 6 years ago

@rgerhards I haven't forgotten about this, just giving the topic some time to cool and let some ideas cook.

When I respond, I plan to respond in smaller one-off remarks targeting specific points. I've come to understand that my longer posts, while intended to help prevent lots of smaller back & forth exchanges on points that could be covered in just a few, were hard to follow (due to their length if nothing else). I suspect I also had a hard time conveying my point as my own attention was probably starting to wander as I got to the end of what I was writing.

Instead, perhaps a series of quick back & forth posts from each of us over a longer period of time (mainly due to TZ differences) will work better.

rgerhards commented 6 years ago

Instead, perhaps a series of quick back & forth posts from each of us over a longer period of time (mainly due to TZ differences) will work better.

yeah ... everything has it's pro and cons. But in my experience up-to-the-point trackers and focused discussions work best on such projects. Except, of course, for some exceptions ;-) But you can't have everything...

rgerhards commented 6 years ago

To get something on the topic: I still think that we should use a dumb-simple solution for the stable releases. The release build script could write the version files and maybe even add them to git. So master will never have them, -stable will always have them.

deoren commented 6 years ago

@rgerhards: I still think that we should use a dumb-simple solution for the stable releases. The release build script could write the version files

That's close to what happens now, though instead of external flat-files, the source/conf.py file is modified by the release script instead. That modified file is then bundled into the tarball for use by anyone building from it (usually package maintainers).

Goals: Recap

Let me make sure I understand the goals here:

As you noted, this means that some work is performed to maintain static values for the v8-stable branch. If you find it acceptable to use a script to perform the maintenance and build of the release content, that opens up multiple options, including the automatic maintenance of the flat-files that you mentioned.

Committing it to the branch could be automated, but the script uses an existing tag to determine the latest version. So in a way, you would be updating the branch to include stable tag + 1 commit for the flat-files.

Compromise

Might this work?

  1. return to hand-editing a set of flat-files
  2. update source/conf.py logic to ignore those files for any branch without "stable" in the branch name
    • v5 and v7 branches do not have the automation logic, and since there are no plans to add it there this effectively turns this into v8-stable vs everything else
    • this approach supports official builds as well as unofficial builds like the ones I perform for demo/prototype work

This opens up the opportunity to refine the process in a future PR to allow automation of the flat-files through the release build script.

deoren commented 6 years ago

@rgerhards: can't we "just" keep the version strings inside a file, then we can check in the conf.py log if that file exists and, if so, use the version from there?

I think your last update to this ticket touched on it, but if we add the files to master then that effectively blocks any branch based on master from using the dev build logic. A workaround is to have all builds use the files unless an environment variable is set to indicate that the dev build logic should be used. That then puts the responsibility into the hands of the build script instead of having the source/conf.py look at placeholder values within the file. Further, this means that instead of external files we just move the static version/release strings back into source/conf.py and maintain by hand there.

Ironically this is something I considered a while back, but was hoping that it wouldn't be necessary. I was hoping that it would be acceptable to consider all non-release builds as unofficial and use the dev build logic for all of them. For the release builds, you could use something (I thought of release_build.sh as a reference script for the release behavior) to generate an official tarball for downstream use and official build content for upload to the main website. This would concentrate all of the "stable release" logic in that one place, unless you opted to setup a cron job on an official build box to maintain the official tarball and stable branch content.

rgerhards commented 6 years ago

That's close to what happens now, though instead of external flat-files, the source/conf.py file is modified by the release script instead.

I overlooked that or better said: I did not think we could use it on git as whole...

That modified file is then bundled into the tarball for use by anyone building from it (usually package maintainers).

I think we are all set if we just git add the modified version. Then all is well. No need for extra files.

I just tried doing that on a test branch but got an error from the script. Will create an issue for that.

rgerhards commented 6 years ago

I think we are all set when modifying conf.py, but just for explanation:

I think your last update to this ticket touched on it, but if we add the files to master then that effectively blocks any branch based on master from using the dev build logic.

I would not add these files to master. That was the core idea. If the file was present, use the values from it; if not, do the "master branch logic". The file itself would have been the flip between the two.

IMHO that would have solved all issues and be extremely easy to implement and maintain. But comitting the changed conf.py I think is still better. Think about it: stable branches are not meant to be updated!