Make it easier to output TOML, YAML, and JSON fragments (possibly with wrapping objects)

[halostatue]

I’m trying to make my chezmoi.toml.tmpl template re-initializable (right now, it loses all of my customizations, because there are complex data structures), and I want to put something like this in:

{{- if (. | dig "github" "tokens" false) }}
{{ .github.tokens | toToml }}
{{- else -}}
# [data.github.tokens.homebrew_api]
# type = "1p"
# name = "***"
# vault = "***"
# account = "***"
{{- end }}

This doesn’t really work well, as it generates:

[github_gh]
account = 'zieglers'
name = 'github-gh-oauth'
type = '1p'
vault = 'Personal'

instead of:

[data.github.tokens.github_gh]
account = 'zieglers'
name = 'github-gh-oauth'
type = '1p'
vault = 'Personal'

Even something fancy ends up doing the wrong thing:

$ chezmoi execute-template '{{- $dict := dict "data" (dict "github" (dict "tokens" .github.tokens)) -}}{{ toToml $dict }}'
[data]
[data.github]
[data.github.tokens]
[data.github.tokens.github_gh]
account = 'zieglers'
name = 'github-gh-oauth'
type = '1p'
vault = 'Personal'

When eventually consolidated, this would result in an invalid TOML file if there are any repeats of [data.github] or [data] anywhere (TOML has a rule of single declarations). The TOML generation of the "blank" top-level pieces (data, data.github, data.github.tokens) is unnecessary.

While the example is focused on TOML, the issue may also exist for generating YAML or JSON fragments for the init template, but each of those may need slightly different approaches (the toYaml may need some sort of indent level for the generated fragment).

Workaround

It’s ugly:

{{- if (. | dig "ruby" "rubygems" false) }}
{{- range $key, $value := .ruby.rubygems -}}
{{-   if (typeIsLike "map[string]interface {}" $value) -}}
[data.ruby.rubygems.{{ $key }}]
{{ $value | toToml }}

{{ end -}}{{- end -}}
{{- else -}}
# [data.ruby.rubygems.geminabox]
# type = "1p"
# name = "***"
# vault = "***"
# account = "***"
#
# [data.ruby.rubygems.apikey]
# type = "1p"
# name = "***"
# vault = "***"
# account = "***"
{{- end }}

[twpayne]

What is the specific suggestion here?

In general, I would suggest two different approaches.

Firstly, if you're going to generate TOML then consider building a dict that has all the values you want in it and only calling toToml at the end, rather than mixing calls to toToml with text. Features like .chezmoidata and template functions like setValueAtPath can help here.

Secondly, if you have this amount of detail in your config file, then that's probably not the best way to do it. Instead, I recommend using a few high-level boolean flags and then using these in your templates instead. For example, in my .chezmoi.toml.tmpl results in my config file being something like:

[data]
    docker = true
    ephemeral = false
    email = "tom.payne@work.com"
    work = true
    headless = false
    hostname = "legion"
    personal = false
    onepasswordAccount = "VBDX..."
    osid = "linux-ubuntu"

Short config files are easy to manage and the per-machine customization is done in templates that use these high-level boolean variables.

[halostatue]

So, I got it working, but I think that the configuration that I have is the smallest that I can make it while keeping personal configuration out of git (the downside to .chezmoidata).

I’m not entirely sure what the specific suggestion is, except that the shenanigans I had to write for this so that I can have a chezmoi.toml that is chezmoi init-safe is extreme, and it feels like something that could be improved upon with targeted functions…but I am not sure how.

Outside of chezmoi init, I can also see cases where I might have a structured config file which is partially chezmoi-managed, but partially volatile and need to make sure that chezmoi apply doesn’t break this.

In this case, the flow would be something where I might have ~/.foo.json and ~/.local/share/chezmoi/private_dot_foo.json.tmpl. To make this work, I would need to:

1. Read the current state of the target file.
2. Merge the configuration data into the read state.
3. Write the merged state to the target file.

This flow isn’t solved by the request for fragment support, but it’s similar to what would be fully required for supporting the load, merge, write functionality that you suggested.

In any case, this is probably better as a Discussion and probably for v3, because it is an extremely advanced use case.

There are additional cases that I can think of where I am using 1Password as both a data source and a bit of a self-direction, so something like a wrapper for op document list and/or op item list possibly with filtering might be a good idea, so that the scripts that I have which are looping over the local configuration could simply be changed to filter over a list of matching documents from 1Password.

Like I said; this is vague but it feels like something that could be fairly powerful for "automatic" discovery.