Add conditions directive/frontmatter and pieces

Is your feature request related to a problem? Please describe.

I have a love-hate relationship with Go templates. I love the simplicity, but hate the verbosity. Nearly every config template I create starts out with a bunch of conditions. A common example is lookPath or hasKey.

Describe the solution you'd like

This is sort of two feature requests but they make the most sense together.

First, what I mean by a conditions directive/frontmatter is it would be great to have a way to more simply list conditions for each component upfront including data variables.

Use case for data variables

I'll start out with the most useful examples and this is accompanies #3326 nicely, but I'll pretend that doesn't get implemented here.

./.chezmoidata/environment/firefox.jsonc

/*
chezmoi:conditions:
  lookPath:
    - firefox
    - sway
*/
{
    "environment": {
        "MOZ_ENABLE_WAYLAND": 1
    }
}

./.chezmoidata/environment/java.jsonc

/*
chezmoi:conditions:
  lookPath:
    - java
*/
{
    "environment" {
        "JDK_JAVA_OPTIONS": "-Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.crossplatformlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dsun.java2d.opengl=true"
    }
}

./private_dot_config/exact_environment.d/managed.conf.tmpl

{{ range $key, $value := .environment }}
{{ key }}="{{ $value }}"
{{ end }}

Use case for templates

This is useful for single templates, but where it could really come in handy is if there was a way to generate a file based on pieces from a folder with templates that each have additional conditionals.

I would like to see this cover two use cases.

1. Numerical Order

The ability to assign a numerical order to each file based on the order in which they should be processed and included. This reminds me a lot of Ansible's assemble module:

./private_dot_config/exact_environment.d/pieces_managed.conf.tmpl

# chezmoi:conditions:
#   lookPath:
#     - systemctl

# Can include text or templated data here that will get processed before any pieces, e.g.:
CHEZMOI_PIECED=1

./private_dot_config/exact_environment.d/pieces_managed.conf/1_firefox.tmpl

# chezmoi:conditions:
#   lookPath:
#     - firefox
#     - sway

MOZ_ENABLE_WAYLAND=1

./private_dot_config/exact_environment.d/pieces_managed.conf/2_java.tmpl

# chezmoi:conditions:
#   lookPath:
#     - java

JDK_JAVA_OPTIONS='-Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.crossplatformlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dsun.java2d.opengl=true'

2. Post tasks

These templates run after numerical pieces have been assembled, and it doesn't matter which order they are ran in. Each one modifies the file, and then passes it on to the next.

./private_dot_config/exact_environment.d/pieces_managed.conf/x11.tmpl

# chezmoi:conditions:
#   eq:
#     env:XDG_SESSION_TYPE: x11

{{ replaceAllRegex "^MOZ_ENABLE_WAYLAND=(.+)$" "${1}0" .chezmoi.pieced }}

In the end, you might end up with something like this:

~/.config/environment.d/managed.conf

CHEZMOI_PIECED=1
MOZ_ENABLE_WAYLAND=0
JDK_JAVA_OPTIONS='-Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.crossplatformlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dsun.java2d.opengl=true'

Describe alternatives you've considered

I've explored other dot file managers and automation tools, but none of them are built for this.

Additional context

The Chezmoi developers provide their wonderful open source tool to the community for free and their time is greatly valued and appreciated. I don't have any expectations that this will get implemented. However, any feedback is welcome so that if someone is interested in doing the work it is good to know in advance if the project maintainers would be open to accepting a PR.

I think that these would be better examined separately, as while they may work well together, the cases for them would need to be made independently as they both have fairly large impact on how chezmoi works.

Condition Directives

This is a major change to how chezmoi directives work. Right now, they're a simple regular expression match with line removal. They're fast, easy to parse, and have zero impact on the output of the templates, but change how the templates engine is configured.

This would be more akin to adding front matter to templates, and would need their own parser. Your examples use a YAML-ish format that would require:

non-greedy parsing (reading possible conditions until we hit something that doesn't look like a condition) or a sigil (chezmoi:endconditions)
parsing the resulting keys and values in ways that may not be directly compatible with the template functions
calling the template functions referenced outside of the context of a template

In your Firefox example, lookPath doesn't work that way (it accepts a single argument), so the custom language described here would have a lot of heavy lifting to perform.

I personally find

/*
chezmoi:conditions:
  lookPath:
    - firefox
    - sway
*/

to be less readable than

{{- if and (lookPath "firefox") (lookPath "sway")) -}}

barring the need for the {{- end -}} at the end of the file, too.

In my dotfiles, I’ve got enough lookPath calls that I've made programs.tmpl that I include:

{{- $programs := includeTemplate "programs.tmpl" | fromJson -}}
{{- if and $programs.firefox $programs.sway -}}
…
{{- end -}}

Pieces / Assemble

This is a fairly good idea, and the addition of another attribute would be a good way to do it, although I am not sure that the top-level template is desirable (and it would have to be a template, as pieces_managed.conf can only be either a file or a directory) compared to pieces_managed.conf/*.tmpl.

This can be manually achieved now, although the files would need to live in .chezmoitemplates/:

# private_dot_config/managed.conf.tmpl

{{ includeTemplate "managed.conf/firefox.tmpl" }}
{{ includeTemplate "managed.confi/java.tmpl" }}
{{ includeTemplate "managed.conf/x11.tmpl" }}

I don't think that pieces is the right terminology, and I'm not sure that this is something that would be required frequently enough to justify the effort for a new file attribute.

Pieces / Assemble

This is a fairly good idea, and the addition of another attribute would be a good way to do it, although I am not sure that the top-level template is desirable (and it would have to be a template, as pieces_managed.conf can only be either a file or a directory) compared to pieces_managed.conf/*.tmpl.

The top-level file may not be ideal or preferred. An alternative could be an _index.tmpl file inside the directory or some other name that gets processed as a template first with each subsequent numbered file/template that gets included and appended to it.

The post tasks was more for use cases where you may need to manipulate the entire file with if type statements.

For now I'm going to try stat and see if it lists the ordered files in the templates directory and build out a template that does this using a corresponding directory in the .chezmoitemplates folder. Here is a rough pseudo example:

{{ if (eq (ext .name) ".tmpl" }}
{{ includeTemplate .name }}
{{ else }}
{{ include .name }}
{{ end }}

I don't think that pieces is the right terminology, and I'm not sure that this is something that would be required frequently enough to justify the effort for a new file attribute.

The term "pieces" was more of a unique play on the "assembled" term that Ansible uses. It could be "assembled" or another word that denotes that it essentially includes sub-templates and appends them to the file.

I agree that I could probably just use a base template to accomplish this and a corresponding path in .chezmoitemplates, but having a way to do this without needing a "duplicate" folder in .chezmoitemplates for each file that I want to assemble would make the workflow much simpler.

This is a major change to how chezmoi directives work. Right now, they're a simple regular expression match with line removal. They're fast, easy to parse, and have zero impact on the output of the templates, but change how the templates engine is configured.

This would be more akin to adding front matter to templates, and would need their own parser. Your examples use a YAML-ish format that would require:

non-greedy parsing (reading possible conditions until we hit something that doesn't look like a condition) or a sigil (chezmoi:endconditions)

parsing the resulting keys and values in ways that may not be directly compatible with the template functions

calling the template functions referenced outside of the context of a template

Yeah, it would definitely not be a simple code change. I'm not sure that it would even be within the goals of the Chezmoi maintainers either, but I honestly find the frontmatter approach much simpler to mentally process upfront about my expectations. In some ways I'm biased towards YAML for this, but it could be some other format that might be easier to parse.

The thing that nags me about Go templates is that with grouping multiple conditions, the parenthesis approach seems to be the most logically ordered but can put you into what I call parenthesis hell, where at the end you may end up with a bunch of ')))))'. Some indention and newlines help, but in the end I just feel dirty and have a bunch of extra newlines just so I can figure out where my parenthesis start and end.

The pipeline approach is easier to read but makes me have to think about everything in reverse, and not all the functions are designed for it nor does it handle several use cases, so you end doing both regardless.

Because of that I've just opted to use parenthesis for everything and I have no ragrets, but there are times where I can't help but say... WTF Go Templates?!?

In your Firefox example, lookPath doesn't work that way (it accepts a single argument), so the custom language described here would have a lot of heavy lifting to perform.

I'm not sure the best way to this. In the list approach for lookPath that I provided, it could automatically add the and statement, but I also envisioned being able to do more than just lookPath, so it would need some planning or a proposal spec I suppose.

Thank you for the kind words and the interesting and clearly presented idea! I also struggle a lot with Go templates, but...

I am strongly opposed to the ideas presented here for multiple reasons:

Condition directives are an example of the inner platform effect. Specifically, this is inventing a new way to write conditional expressions when chezmoi already supports conditional expressions via Go templates. As demonstrated with the lookPath example, the interpretation of this language is under-specified and will require a large amount of magic (or heavy lifting as @halostatue calls it).
Implementing pieces_ will require a lot of specialized code to handle, but, more importantly, will be difficult to use because the user will have to reason about multiple, possibly conflicting, files simultaneously.

As an alternative, consider how the ./private_dot_config/exact_environment.d/managed.conf.tmpl example looks using chezmoi's existing mechanisms:

CHEZMOI_PIECED=1

{{- if or (lookPath "firefox") (lookPath "sway") }}
MOZ_ENABLE_WAYLAND={{ ne (env "XDG_SESSION_TYPE") "x11" | int }} 
{{- end }}

{{- if (lookPath "java") }}
JDK_JAVA_OPTIONS='-Dawt.useSystemAAFontSettings=on -Dswing.aatext=true -Dswing.crossplatformlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dswing.defaultlaf=com.sun.java.swing.plaf.gtk.GTKLookAndFeel -Dsun.java2d.opengl=true'
{{- end }}

This is a single short file that clearly communicates what the contents of ~/.config/environment.d/managed.conf should be.

For more complicated examples, as @halostatue points out, chezmoi already has the include and includeTemplate functions that allow you to assemble multiple files.

As another alternative, if the program that reads the files in ~/.config/environment reads all the files in ~/.config/environment, then you can create separate files in ./private_dot_config/exact_environment.d for each variable that you want to set.

As a side-rant, I've had the misfortune of having had to write Ansible playbooks in the past, and Ansible's attempt to use YAML as a programming language was (and maybe still is?) and unmitigated disaster. Specifically, YAML is a data declaration language and does not have control flow. Ansible needs control flow and so tries to add it to YAML, and the result is the exact Inner-platform effect: a weak, under-specified, language with no tooling. As much as Puppet's declarative approach was a strong source of inspiration for chezmoi, Ansible is the exact opposite of what I want chezmoi to be.

So, thank you again for the well-considered proposal, but no.

The thing that nags me about Go templates is that with grouping multiple conditions, the parenthesis approach seems to be the most logically ordered but can put you into what I call parenthesis hell, where at the end you may end up with a bunch of ')))))'. Some indention and newlines help, but in the end I just feel dirty and have a bunch of extra newlines just so I can figure out where my parenthesis start and end.

That's why I pointed out the $programs import template approach that I use. In some ways, I did not need that (because chezmoi caches lookPath results), but in others I absolutely did. I can now do:

{{ if or $programs.in2csv $programs.soffice $programs.xlsx2csv $programs.xlsx2csvPy -}}

From home/proviate_dot_config/git/attributes.tmpl:75.

No parenthesis in sight, and I think fairly understandable. Maybe I should change the name I use from $programs to $has (I often write a bash function has() { command -v "$@" >/dev/null 2>&1; }) or something similar, but the name isn't the hardest part here. In your case {{ if and $programs.firefox $programs.sway }}.

twpayne / chezmoi